Prompt caching
Also: prompt cache
Reusing some work on input the model has seen before. When supported, this can cut time or cost.
The fuller definition
Prompt caching reuses computation associated with repeated prompt content when a provider or runtime supports it, potentially reducing repeated processing cost or delay.
2,000
8,000
4,000
2,000
A hypothetical 16,000-token budget. Real limits and token accounting depend on the model. Saving a file outside this budget does not mean it is present in the current context, and a passage inside it is not guaranteed equal attention.
A concrete example
Repeated requests share a stable instruction prefix while varying the final document question.
The distinction that matters
Caching is not permanent memory or a guarantee of identical answers; eligibility, storage, and accounting vary.
Related terms
Context window
How much text the model can hold in mind at once: your question, the documents you added, and the conversation so far. Anything outside it is invisible to the model.
APIs & softwareLatency
The wait between sending a request and getting the answer back.
APIs & softwareAPI — Application Programming Interface
A doorway that lets one program talk to another. Apps use an API to send your request to the model and get the answer back.
Where it is taught
Give AI a brief it can actually followSource and further reading
Anthropic · Context engineeringThe definition and example are written for this library. The link gives background, official documentation or legal authority as labelled. Legal scope depends on the jurisdiction and current text; product features depend on release and settings.