Shift Note: Cache Reads Changed the Token Bill

A request arrives with the same long instructions as the previous request, but its input charge looks different. The likely explanation is prompt caching. Some of that text may have been stored for reuse, allowing the next request to receive a different rate for the repeated portion. To understand the bill, you need to separate ordinary input, cache writes, cache reads, and output.

Repeated instructions can take a different billing path

Models process text as tokens: pieces of words, punctuation, and other content. An API request can contain instructions, reference material, conversation history, and the latest question. Without an applicable cache hit, that content is generally billed at the model’s ordinary input rate.

Prompt caching lets a provider reuse processing associated with an eligible, repeated beginning of a request. That beginning is often called the prompt prefix. It might contain a support handbook or the instructions that your coding assistant sends on every turn.

Caching does not mean the provider simply returns an old answer. The model still handles the current request and generates output. Cached input and generated output remain separate billing categories.

A cache write prepares content for later requests

A cache write happens when eligible prompt content is stored for potential reuse. Depending on the provider, this may happen automatically or require explicit cache settings. Some pricing schemes charge a separate write rate, which can be higher than ordinary input. Others handle cache creation without a separate premium.

This matters because the first request may carry an extra cost before any later request benefits. If you cache a large document and never reuse it, there may be no saving to recover that cost. A cache write creates an opportunity for cheaper reuse; it does not guarantee a lower total bill.

A cache read discounts the portion that matches

A cache read occurs when a later request can reuse eligible cached content. That portion may be billed at a lower input rate. New text still receives its applicable input rate, and newly generated tokens receive the output rate.

Suppose your assistant sends 10,000 tokens of stable instructions followed by a new 500-token question. If all 10,000 instruction tokens qualify for a cache read, the request could contain 10,000 cached input tokens plus 500 ordinary input tokens. The response adds a separate output charge. This is an illustrative token count, not a promise that every provider will cache that amount.

A cache hit does not make the whole request free. It changes the billing treatment of the portion successfully reused.

The bill depends on reuse before the cache expires

The practical calculation is whether savings from later reads outweigh any extra cost of creating or retaining the cache. Check the current pricing and caching rules for the exact model and service you use.

• Frequent requests with a large, stable prefix offer more opportunities to benefit from cache reads.
• Content that changes on every request can reduce the amount eligible for reuse.
• A gap between requests can allow cached content to expire before it is needed again.
• Minimum token requirements and supported request formats can determine whether caching applies at all.

Keep stable instructions and shared reference material near the beginning of the prompt when the provider’s caching rules call for prefix matching. Put changing questions afterward. Avoid inserting timestamps or random identifiers ahead of reusable content unless the application needs them there.

Usage records explain savings better than request counts

Two requests can have similar total token counts and different costs because their cached portions differ. Inspect the usage fields and billing breakdown available through your service. Look for ordinary input, cache creation or writes, cached input or reads, and output. Field names and reporting detail vary.

When comparing models, compare the cost of your actual request pattern. A low cache-read price is useful only when your workload produces qualifying reads often enough.

Check the cache terms before the next integration

Relay365 is a practical option if you want pay-as-you-go access to multiple model families through one relay service. It works with Claude Code, Codex, Gemini CLI, and OpenAI SDK clients. Before choosing a route, check its current model pricing, cache support, and usage reporting rather than assuming every model handles caching identically.

Create a Relay365 account and test a small sequence of repeated requests, then inspect the recorded usage and charges. That gives you concrete evidence of whether your application’s repeated input is producing worthwhile cache savings.