AWS lays out prompt caching for OpenAI models on Bedrock: combine implicit and explicit breakpoints, with a 1,024-token minimum prefix
Agents re-read their whole history every turn, so caching is where the savings are. These are caching rules you can copy.
// Key points
- Bedrock supports implicit caching, where it chooses where the reusable prefix ends, and explicit caching marked with prompt_cache_breakpoint. AWS recommends both: explicit for tools and instructions, implicit for the growing history.
- The reusable prefix must reach 1,024 tokens. In one test, a roughly 4,000-token tools-and-instructions prefix wrote 4,120 tokens on the first request and read those 4,120 tokens on later ones.
- Caching only hits on identical text: serialize tools and instructions the same way every time, keep MCP-delivered tools in a stable order, and put variables such as the date, user name or request ID after the cached content.
- store=False only stops response storage on the service side; it does not disable prompt caching.
Builder's takeThese rules apply well beyond Bedrock: putting a timestamp at the top of the system prompt is the most common cache-buster I see. In my post on a month of real RAG bills, controlling context length paid back the most, and agents are the same: make tool definitions and instructions a fixed prefix, watch the cache hit rate, and only then think about switching models.
// Background · from #AI agents
Full timeline →- Oct 27 GPT-6.1 Terra is generally available on Amazon Bedrock the same day; OpenAI's evaluation has it finishing tasks over 3x faster than GPT-6 Astra
- Oct 27 Cloudflare discloses a GLM-5.3 cross-request leak: 155 responses carried other customers' data, caused by a KV pool smaller than the context window plus an SGLang bug
- Oct 27 Google Docs edits Markdown files natively and Drive previews them, aimed at humans and agents working together