Cache Reads and Writes: Why They Change the Bill

#1 · Start with what prompt caching actually reuses

When you call a language model, you often send the same material repeatedly: system instructions, tool definitions, a reference document, or earlier conversation turns. Prompt caching lets a supported service reuse processing associated with an eligible portion of that input.

It is not usually a store of finished answers. A cache hit does not mean the model skips generating a response, and it does not make output tokens free. The main billing change concerns how reusable input tokens are classified and priced.

#2 · Separate the billing categories

A useful mental model has four buckets:

• Ordinary input: input tokens processed without a cache discount.
• Cache writes: tokens used to create or refresh a reusable cache entry, where separately billed.
• Cache reads: tokens served from an eligible existing cache entry.
• Output: tokens the model generates in its response.

Not every provider exposes all four buckets. Some report cached input as a subset of total input; others expose separate creation and read fields. Do not add overlapping counters together. Read the billing definitions before turning usage data into a cost estimate.

#3 · Understand why the first request can cost more

A cache write prepares reusable input for later requests. Depending on the provider, model, and retention option, that write may carry a different rate from ordinary input. A later cache read may have a lower rate, but the initial setup is not automatically free.

This is why caching is a reuse strategy, not an unconditional discount. If you submit a long document once and never reference it again, a separately charged cache write might not pay for itself.

#4 · Walk through three requests

Consider a hypothetical workload using Claude Haiku 4.5, subject to the cache behavior supported by your chosen endpoint. Each request contains a stable 10,000-token reference section and 1,000 tokens of new instructions. Each response generates 500 tokens.

Assume the reference section is eligible, the first request writes it, and the next two requests read it before expiry. Also assume the billing categories below are mutually exclusive:

• Without caching: 33,000 ordinary input tokens and 1,500 output tokens.
• With caching: 3,000 ordinary input tokens, 10,000 cache-write tokens, 20,000 cache-read tokens, and 1,500 output tokens.

These are illustrative token counts, not published prices or a promise of cache eligibility. To compare bills, multiply each category by its applicable rate, using consistent units such as price per million tokens. The output cost is unchanged in this example; the savings depend on whether the two cheaper reads outweigh any extra write cost.

#5 · Give the cache something stable to reuse

Put reusable instructions and reference material before request-specific content when the provider uses prefix-based caching. Keep that prefix consistent. Moving a timestamp, request ID, or changing instruction to the beginning can prevent later content from matching.

Check minimum eligible lengths, supported content types, retention rules, and whether caching requires explicit controls. A request sent after expiry may need another write. Similar meaning is not enough: matching requirements are provider-specific and can be sensitive to exact content and ordering.

#6 · Check the bill, not just the hit counter

Record ordinary input, cache creation, cached input, and output usage where available. Compare a representative workload with and without caching rather than judging one successful hit. Include failed reuse attempts and repeated writes.

Also check whether your SDK or relay preserves the required cache controls and usage fields. OpenAI-compatible request formatting alone does not establish identical caching behavior across model families. Confirm the selected route's pricing and accounting conventions.

#7 · Choose a route that makes comparison practical

Relay365 can fit teams that want pay-as-you-go access to multiple model families through a relay desk, including workflows using Claude Code, Codex, Gemini CLI, or an OpenAI SDK. Its API base is https://api.api365relay.com; OpenAI clients add /v1. Treat cache support and rates as route-specific details to verify, not assumptions.

If that setup suits your workflow, create a Relay365 account and start with a small, measurable repeated-prompt test. The goal is straightforward: verify that reusable input becomes cheaper reads often enough to justify the writes.