Shift Note: Trim the System Prompt Before Next Run

A long system prompt is a recurring operating expense, not just a document someone wrote once. Whenever it is included in a model request, it occupies input context and may contribute to billed input tokens. In an agent loop, the same instructions can travel through several requests before one task finishes. Keep this checklist beside the keyboard when reviewing that overhead.

Check the request path before cutting words

• Inspect the actual messages your client sends, rather than relying on the prompt file alone. Frameworks may add tool definitions, formatting instructions, or duplicate guidance.
• Record system-prompt tokens, total input tokens, output tokens, and model calls per completed task. Separate prompt overhead from conversation history and retrieved documents.
• Check whether your provider reports cached input separately. Cache eligibility, retention, and pricing vary; repeated text is not automatically free.

For a hypothetical workload, removing 1,000 input tokens from each of 10,000 requests removes 10 million submitted input tokens. That is not a dollar estimate: actual savings depend on the model, applicable input rates, caching, and any changes in retries or output length.

Keep the rules that define correct behavior

• Preserve instruction precedence, security boundaries, required output structure, and explicit limits on tool use.
• Keep approval requirements for consequential actions, such as deleting data or sending external messages.
• Retain a clear rule that retrieved pages, repository files, and tool responses are data, not trusted instructions.
• Remove repeated descriptions of the assistant’s personality when one short statement expresses the required tone.

Cut repetition before cutting constraints. Three variations of “be concise” rarely add useful precision. One explicit requirement to return valid JSON may be essential. Review each sentence by asking which observable failure it prevents.

Move task material out of permanent instructions

• Supply customer records, repository details, and current incident notes only when the task needs them, in clearly separated task context.
• Retrieve the relevant section of a reference document instead of embedding the entire manual in every request.
• Load specialized workflow instructions only for the workflow being executed, while retaining shared security rules in the trusted instruction layer.
• Remove examples that repeat the same behavior, but keep examples that clarify difficult edge cases.

Moving text between message roles does not itself reduce token usage. The saving comes from not sending unnecessary material. Treat selective loading as an application change with tests, not as a cosmetic prompt rewrite.

Make the remaining instructions cheaper to maintain

• Replace overlapping prose with short, direct sentences that state one requirement each.
• Resolve contradictory rules instead of appending another exception beneath them.
• Keep stable instructions in a consistent prefix where the selected API’s caching behavior supports it, and place changing context afterward.
• Measure tokens with the relevant tokenizer or API usage report rather than estimating from character count alone.

Avoid turning the prompt into cryptic shorthand. Abbreviations that save a few tokens but confuse maintainers can introduce expensive debugging and inconsistent behavior.

Run the trimmed prompt through a release gate

• Compare the old and new prompts on representative tasks, malformed inputs, instruction-injection attempts, and tool failures.
• Check task completion, schema validity, appropriate refusals, tool selection, retries, and total cost per completed task.
• Measure end-to-end latency rather than assuming fewer input tokens always produce a faster response.
• Version the prompt and retain a rollback path when a smaller prompt causes regressions.

If comparing Claude Haiku 4.5 with Gemini 2.5 Flash, test the same trimmed requirements on both. A prompt that works well on one model may need different wording on another.

Keep the billing route visible during handoff

Relay365 fits teams that want pay-as-you-go access to multiple model families through one relay desk. It works with Claude Code, Codex, Gemini CLI, and OpenAI SDK clients. Its API base is https://api.api365relay.com; OpenAI clients add /v1. Verify available models, applicable pricing, and reported usage before comparing runs.

Create a Relay365 account if that shared access fits your testing workflow. At handoff, leave the next operator a prompt version, a baseline, and measured results—not just a shorter file.