prompt caching

Definition

Prompt caching is the LLM provider's optimisation of reusing computation across requests that share a long, stable prefix. Anthropic and OpenAI both offer it. The cache hit returns at a fraction of the input-token price and lower latency. Designing prompts so the cacheable prefix (system prompt, tool defs, RAG docs) comes first is the key lever for cost on production agents.

When to use

See also

• LLM — Large Language Model — a transformer-based model trained on internet-scale text to generate and reason.
• agent — An LLM-driven program that plans, calls tools, observes results, and loops until it reaches a goal.