Context Windows: What Actually Fits Inside

#1 · Think of context as a working desk

A model’s context window is the amount of information it can work with during a request, measured in tokens. It is not permanent memory, and it is not simply the maximum size of a document you can upload. Instructions, conversation history, retrieved passages, tool results, and other inputs can all occupy space. Depending on the model and API, generated output and reasoning tokens may also share that budget.

The practical question is not “How large is the window?” It is “How much room remains for my material after everything else is included?”

#2 · Learn what a token measures

Tokens are pieces of text, not a universal word count. A token might represent a word, part of a word, punctuation, or whitespace. English prose, source code, tables, and different languages can tokenize differently.

For rough planning, English prose often averages around four characters per token, but that is an estimate, not a conversion rule. Use a tokenizer compatible with your chosen model, or a token-counting endpoint when available. Do not assume a PDF’s file size tells you its token count: extraction can introduce repeated headers, broken tables, or other clutter.

#3 · Build a budget before sending anything

Imagine a hypothetical model with a 128,000-token combined context limit. This is a budgeting example, not a specification for any particular model. Your request might need:

• 4,000 tokens for system instructions and tool definitions.
• 12,000 tokens for conversation history.
• 8,000 tokens reserved for the response.
• 4,000 tokens of safety margin for formatting and unexpected overhead.

That leaves about 100,000 tokens for documents and other source material. Actual accounting varies by API. Check whether input and output limits are separate, whether reasoning consumes an additional allowance, and whether your configured output maximum reduces the input budget.

#4 · Count the parts you do not normally see

A chat interface may display a short conversation while sending a much larger request. Coding agents can include repository instructions, file contents, command output, and tool schemas. A retrieval application can attach several passages for every question.

Images, audio, and video also consume capacity under model-specific rules; their cost cannot reliably be estimated with an English text heuristic. Inspect request payloads and usage reports where available. In Claude Code, Codex, or Gemini CLI, also check how the client handles history, compaction, and automatic file inclusion.

#5 · Separate “fits” from “works well”

A request can fit within the limit and still produce an incomplete answer. Relevant details may be scattered across unrelated documents, contradicted by outdated notes, or buried in repetitive logs. A larger window increases capacity; it does not guarantee that every detail will be used correctly.

Give the model a clear task, identify authoritative sources, and label document boundaries. Ask for evidence tied to filenames or section labels. For important workflows, test questions whose answers occur in different parts of the material rather than judging performance from one successful summary.

#6 · Choose a model and route deliberately

Compare the exact context limit, output cap, supported input types, pricing, and endpoint behavior for the model you will actually call. If you are evaluating Claude Haiku 4.5 and Gemini 2.5 Flash, verify their current documentation rather than inferring capacity from their names.

Relay365 is a pay-as-you-go relay desk for many model families and can be useful when testing the same workload across providers. It works with Claude Code, Codex, Gemini CLI, and OpenAI SDK clients. Still, confirm the model and route limits before moving a large-context workflow into production.

#7 · Keep less, preserve more value

Start with relevant excerpts instead of entire archives. Remove duplicated boilerplate, cap verbose tool output, and summarize older conversation turns while preserving decisions and unresolved questions. Keep original sources available so summaries can be checked.

If comparing models through one relay fits your workflow, create a Relay365 account and begin with a small, representative test. Measure token usage, response quality, latency, and cost. The best context window is not necessarily the largest one: it is the one that reliably supports your task with a budget you understand.