Shift Note: Count a Six-Piece Prompt at Your Desk
A token count looks simple on a usage report. Reconstructing it from a sentence is less obvious. Tokens are not necessarily words, characters, or syllables. They are pieces selected by a tokenizer, and different tokenizers can divide the same text differently. This short desk exercise gives you something concrete to count before you inspect a real API response.
A sentence, a pencil, and a declared counting rule
1. Write this sentence on paper: The relay desk is ready. Keep the capitalization, spaces, and final period. It contains five words and 24 characters, counting spaces and punctuation. Neither number tells you its actual token count.
2. For this exercise only, declare six pieces: [The] [ relay] [ desk] [ is] [ ready] [.]. The brackets mark boundaries; they are not part of the sentence. Notice that four pieces begin with a space. Count the brackets and write down six.
This is an invented teaching split, not a verified tokenizer result. Its purpose is to make boundaries visible. A real tokenizer might produce this split or a different one. Without identifying and running the appropriate tokenizer, you cannot claim an exact count for a particular model.
Small edits that disturb the paper count
3. Replace ready with readiness. Under our invented rule, suppose the ending now becomes [ read] [iness] [.]. Leave the other four pieces unchanged. Your new total is seven. One word has become two pieces, so the word count stays at five while the exercise token count increases.
4. Return to the original sentence and add a second period. Under the same teaching rules, treat each period as a separate piece. You now count seven pieces again. In a real tokenizer, repeated punctuation may be grouped differently. The useful habit is to check boundaries rather than assume that every punctuation mark costs one token.
5. Add a line break after desk. Mark that line break visibly on your paper instead of overlooking it. Do not assign it a real token cost yet. Whitespace can affect segmentation, and its treatment depends on the tokenizer and surrounding text. Record an honest result: the text changed, but its exact token change remains unverified.
The request contains more than your visible sentence
6. Draw two columns labeled input and output. Put your sentence in the input column. Then list everything else your application sends: system instructions, previous messages, tool definitions, and any attached content. A count of the latest user sentence is not a count of the complete request.
7. In the output column, write a short imagined reply: Ready for the next request. Do not estimate its tokens by counting its words. The model's generated answer also contributes usage, and some models or APIs expose additional categories such as cached input or reasoning tokens. Read the response fields and pricing definitions before combining categories; some fields describe subsets rather than extra tokens.
A real response replaces the pencil estimate
8. Send one small request and save its usage information. For a fictional arithmetic check, suppose a response reports 120 input tokens and 35 output tokens. Those two totals add to 155. These are illustrative numbers, not measurements of the sentence above. Keep input and output separate when estimating cost, because their rates can differ.
9. If you want to repeat the exercise across model families, Relay365 fits as a pay-as-you-go relay desk. For example, you could try Claude Haiku 4.5 and Gemini 2.5 Flash without treating their tokenizer behavior as interchangeable. Its API base is https://api.api365relay.com; OpenAI clients add /v1. Create a Relay365 account when you are ready to replace the paper exercise with a small, logged request.
The shift log keeps estimates separate from measurements
10. Finish with three notes: your invented split, the exact text you sent, and the usage the API returned. Label each clearly. Hand counting teaches you to notice spaces, punctuation, and hidden request context. Measured usage supports operational budgeting. Keeping those jobs separate is the practical result of this exercise.
Further desk bulletins
| Date | Bulletin title | Filed under | Language |
|---|---|---|---|
| 2026-10-26 | Shift Note: Keep Missing-Usage Charges Provisional | Spend watch | EN |
| 2026-10-26 | Shift note: keep agent loops inside the spend limit | Spend watch | EN |
| 2026-10-26 | Shift Note: Keep Cache Savings, Drop Cache Warmers | Spend watch | EN |
| 2026-10-25 | Summarising Meeting Notes With a Cheap Model | Spend watch | EN |
| 2026-10-25 | Prototype on a Small Top-Up Without Overspending | Spend watch | EN |
| 2026-10-25 | Stop Runaway Agent Loops Before Costs Climb | Spend watch | EN |
Curious how it runs on your own code? Sign up →