Prompt Caching: How Cache Reads Lower the Bill

A coding agent repeatedly sends repository instructions, tool definitions, and conversation history. Prompt caching can reuse processing for an unchanged prefix instead of processing that entire prefix again. The financial benefit comes from eligible tokens being billed at a lower cache-read rate, not from making the whole request free. Here is a practical walkthrough. ([developers.openai.com](https://developers.openai.com/api/docs/guides/prompt-caching?utm_source=openai))

1. Identify the reusable prefix

Start with a request your application actually repeats. Separate stable material, such as coding standards and tool schemas, from changing material, such as the latest question or patch. Put stable content first and changing content afterward. Cache matching depends on the rendered prefix matching exactly; equivalent wording is not enough. Changes to tool definitions or relevant settings can also prevent reuse. ([developers.openai.com](https://developers.openai.com/api/docs/guides/prompt-caching?utm_source=openai))

Give the stable template a version identifier in your application logs. That makes an unexpected drop in reuse easier to investigate after a deployment.

2. Confirm the caching contract

Do not treat an OpenAI-compatible request shape as proof that every upstream caching feature behaves identically. Through CodeRelay Hub, verify which caching controls and usage fields your selected route exposes before relying on savings.

For a route serving Claude Sonnet 5.5, the native Claude API documents automatic caching through a request-level cache_control field and explicit breakpoints on content blocks. The placement determines the reusable boundary. Confirm that your integration preserves the intended controls rather than silently dropping them. ([platform.claude.com](https://platform.claude.com/docs/en/build-with-claude/prompt-caching))

3. Separate writes from reads

The first eligible request establishes cached state; later matching requests can read it. Claude distinguishes cache creation from cache reads, and its write pricing depends on the selected lifetime. A cold request therefore does not demonstrate the economics of a warm workload. ([platform.claude.com](https://platform.claude.com/docs/en/build-with-claude/prompt-caching))

Run a small controlled test: send the stable prefix, then send another request with that prefix unchanged and a different question afterward. Inspect usage rather than assuming that a fast response proves a hit. Keep the model and request settings fixed during this test.

4. Calculate the bill by category

Use the billing categories your route actually reports. For a ledger with separate token buckets, calculate:

Total cost = uncached input cost + cache-write cost + cache-read cost + output cost + applicable storage or other charges.

Multiply each token count by its corresponding rate, using consistent units. Some response formats report cached tokens inside an overall input count rather than as an additional bucket. Check the schema before summing: counting the same tokens twice will exaggerate your estimate. OpenAI exposes cached-token details, while Claude reports separate cache-creation and cache-read fields. ([developers.openai.com](https://developers.openai.com/cookbook/examples/prompt_caching101?utm_source=openai))

5. Measure savings across the workload

Compare observed cost with a baseline that prices the same input at the applicable uncached rate. Include cold starts, writes, misses, and any retention charges. Do not apply the cache-read discount to output tokens; output remains a separate billing category. ([platform.claude.com](https://platform.claude.com/docs/en/build-with-claude/prompt-caching))

Track token reuse alongside request hits. A request that reuses only a short prefix should not count as equally valuable as one that reuses most of its input. Also compare cost per completed task, including retries, rather than celebrating a cheaper individual call.

6. Debug misses systematically

Compare the actual serialized requests, not just the template source. Check for timestamps near the beginning, reordered tools, modified schemas, and rewritten history. OpenAI documents exact-prefix matching and compatible settings as prerequisites for reuse. ([developers.openai.com](https://developers.openai.com/api/docs/guides/prompt-caching/diagnostics?utm_source=openai))

Check timing and eligibility too. Cache lifetime and minimum cacheable length vary by model and configuration; retaining a conversation identifier alone does not guarantee a hit. ([developers.openai.com](https://developers.openai.com/api/docs/guides/prompt-caching?utm_source=openai))

Log a template hash rather than sensitive prompt text when possible. Keep upstream route changes visible so you can distinguish a prompt regression from an infrastructure change.

7. Turn the experiment into a guardrail

Build a recurring check around one representative workload. Record input, cached input, writes where available, output, billed cost, and successful task completion. Alert on unexpected cost changes after template or routing updates.

For a pay-as-you-go integration, the goal is not the highest hit count. It is a lower verified bill for useful work, without padding prompts or compromising correctness.

Open a free workspace in CodeRelay Hub.