Prompt caching stores a repeated prefix such as shared instructions or a long document for reuse on a later request. Reuse depends on provider-specific cache conditions and lifetime.
The demo contrasts a first-request miss with a later hit on the same prefix. Actual savings vary with token mix, request timing, and provider policy.
When to use
Use it when many requests share a long instruction or document prefix.