What happened
AI prompt caching is supposed to make repeated API calls dramatically cheaper — but a single line of code can quietly turn that discount off, and most teams never notice. A developer testing this across DeepSeek, OpenAI, and Anthropic found that dropping one dynamic element, like a live timestamp, into the first lines of a system prompt was enough to disable caching entirely on every single call.
Here's why it matters so much: cached tokens are dramatically cheaper than regular ones. As of August 27, 2026, pricing looked like this:
- DeepSeek v4-flash: $0.44 per million tokens normally, $0.014 per million cached — a 31x difference
- OpenAI gpt-5.6-sol: $4.00 per million normally, $0.40 per million cached — 10x
- Anthropic Sonnet: $3.00 per million normally, $0.30 per million cached — 10x
DeepSeek and OpenAI enable caching automatically. Anthropic requires you to opt in explicitly with a `cache_control: {"type": "ephemeral"}` parameter on the relevant content block — skip it, and there's no caching at all, full stop. But once caching is turned on, all three providers break it the exact same way: silently, and from a single mismatched token.
### The mechanism behind the break
Prompt caching works like a prefix tree over tokens. If the first N tokens of your request match the first N tokens of a previous request, those N tokens are served from cache. The moment the sequences diverge — even by one token — caching stops for everything after that point, and the rest of the prompt is billed at full price.
The classic culprit is something like `f"Current time: {datetime.now()}.\n{POLICY}\n{KNOWLEDGE_BASE}"`. The timestamp changes on every call, so the prefix breaks by the fourth word — and the entire prompt behind it, potentially thousands of tokens of policy text and knowledge base content that never actually changed, gets billed as if it were brand new every time.
Why it matters
The cost impact compounds fast at production volume. For an agent running a ~1,200-token system prompt at 10,000 calls a day, the difference between a broken cache and a 98% hit rate looks like this:
- DeepSeek v4-flash: $158/month without caching vs. $8/month with it — a $150 monthly gap
- OpenAI gpt-5.6-sol: $1,440/month vs. $173/month — a $1,267 gap
- Anthropic Sonnet: $1,080/month vs. $130/month — a $950 gap
A live test against DeepSeek made the effect concrete. With a timestamp placed before the static system content ("live-broken"), eight consecutive calls returned a cache hit count of exactly zero, every time, despite prompt sizes staying nearly identical (1,177–1,180 tokens). After moving the timestamp to the end of the prompt ("live-fixed"), the very next calls hit cache for 1,152 of roughly 1,180 tokens — a jump from 0% to about 98% reuse.
That's a deliberately worst-case scenario, since real prompts rarely put dynamic content in the very first line. In more typical production setups, baseline cache-hit rates tend to land around 49–66% rather than 0% — still a meaningful amount of money left on the table, just harder to spot.
### It's easy to fix, if you can see it
If your system prompt starts with `datetime.now()` or anything similar, you already know the fix: move dynamic content — timestamps, session IDs, live counters — out of the front of the prompt and put static content (policies, knowledge bases, tool schemas) first. That single reorder alone can be worth hundreds or thousands of dollars a month.
How to use it today
The timestamp bug is the easy one — it's visible in the code the moment you look for it. The harder ones aren't:
1. Tool schema ordering. Anthropic's cache covers `tools`, then `system`, then `messages`, in that order — meaning tool schemas sit at the very front of the prefix. If you're building that schema list from a Python dictionary with non-deterministic key order, you can break the cache before the system prompt is even reached, and nothing in your prompt code will show it.
2. Knowledge bases built from a `set()`. Same content, different iteration order between processes. The text looks identical to a human, but the token sequence doesn't match, so the cache never fires.
3. Cache reads at zero despite a correct prefix. If `cache_read_tokens` comes back as 0 even though your prompt construction is provably stable, the problem usually isn't your prompt — it's an expired TTL or an upstream routing issue, which needs a fix at the infrastructure level, not the prompt level.
4. Templates that render conditionally. A rarely-triggered conditional block — one that fires once every hundred calls — quietly changes the prefix just often enough to tank your average hit rate without ever showing up in casual testing.
The pattern across all four: the prompt is large, the change is small, and it's invisible by eye. Checking for it means logging the raw `usage` object your provider already returns on every call — `prompt_tokens`, `cache_read_tokens` or equivalent — and watching for hit rates that don't match what your prompt structure should produce. No extra API calls, no data leaving your machine, just re-reading data you're already paying for. Teams that don't want to build this logging by hand can lean on ready-made checkers; browsing a directory like [mykreatool.com](https://mykreatool.com) for free AI utilities is a fast way to find one instead of writing it from scratch.
### What good logging looks like
A minimal version just needs one JSON line per call, capturing the model name, a session ID, and the raw usage object exactly as the provider returned it — no reformatting, no assumptions about field names. Once you have a few hundred of those lines, computing an average hit rate and flagging sessions that dip well below it takes a handful of lines of code.
Who benefits
This matters most for anyone running an AI agent, chatbot, or support tool with a large, mostly-static system prompt — policy documents, knowledge bases, tool definitions — called repeatedly throughout the day. Customer support bots, coding assistants, internal RAG agents, and any high-volume automation built on DeepSeek, OpenAI, or Anthropic models are all exposed. The bigger the system prompt and the higher the call volume, the bigger the invisible bill from a broken cache. Solo builders and startups running lean on API costs stand to save the most in relative terms, since a few hundred dollars a month can be the difference between a side project and a real budget line.
Risks
Caching isn't free of trade-offs. Cache entries expire on a TTL, so bursty or low-frequency traffic may never build up enough hits to matter. Restructuring a prompt to put dynamic content last can also change how a model weighs that information if it's meant to be emphasized — test outputs after reordering, don't assume behavior is unaffected. And logging raw `usage` and `messages` data, even locally, means handling potentially sensitive prompt content responsibly; keep those logs off shared systems unless you've reviewed what's in them.
Conclusion
Prompt caching can cut AI API costs by 10x or more, but it breaks the instant a prefix stops matching — and most of the ways that happens are invisible unless you're actually reading the usage data your provider sends back on every call. Start by checking whether your system prompt puts anything dynamic before the static content, then log `cache_read_tokens` for a week and see what your real hit rate looks like. For high-volume agents, that's often the single cheapest optimization available.
Comments 0