Shift Log: Long System Prompts Lost the Cost Check

The duty-desk question is straightforward: does a longer system prompt earn back its operating cost? Put a policy-heavy prompt beside a compact instruction contract, run both against the same workload, and compare more than the invoice. A cheaper request that breaks authorization rules is not a saving. A longer prompt that repeats itself without improving outcomes is not insurance.

The baseline: a policy binder against an instruction contract

The long version often contains everything accumulated during development: role descriptions, repeated warnings, formatting examples, tool instructions, product documentation, and patches for individual failures. The trimmed version keeps essential behavioral boundaries, authorization constraints, tool rules, and output requirements. Task-specific information arrives only when the task needs it.

For an illustrative calculation, suppose trimming removes 6,000 input tokens from each of 10,000 requests. That eliminates 60 million input tokens before accounting for caching or differences between requests. Multiply the applicable token categories by their actual rates. Do not treat this arithmetic as a price quote: model pricing, cache treatment, and billing details determine the money saved.

Verdict: The compact contract wins on uncached input volume. Whether it wins overall depends on task quality, retries, output length, and the engineering work needed to maintain it.

Repeated instructions lose to explicit, testable rules

A long prompt may say “be concise” in several places while also demanding exhaustive explanations. Repetition does not resolve that conflict. A shorter instruction such as “Return the requested fields only; omit commentary unless requested” is easier to test and maintain.

Start by grouping instructions by purpose. Remove duplicates, replace vague adjectives with observable behavior, and delete examples that demonstrate the same rule. Keep examples when they clarify a genuinely ambiguous requirement. Shortness alone is not the objective; removing information the model needs can increase failures and retries.

• Keep authorization boundaries and tool-use constraints explicit rather than relying on an example to imply them.
• Move changing product facts into task context or a retrieval layer instead of embedding an entire handbook.
• Treat retrieved documents and user-provided content as data, not as instructions allowed to override system rules.

Verdict: Clear rules beat repeated prose. Delete redundancy first, not safeguards.

Cached bulk competes with genuinely smaller requests

A stable prefix may qualify for prompt caching where the provider and route support it. That can reduce the cost of repeated input and may improve latency. It does not make a large prompt universally free, nor does it remove its context footprint. Cache eligibility, retention, minimum sizes, and billing behavior vary.

Compare four cases: long and cold, long and warm, trimmed and cold, trimmed and warm. Where supported, keep stable instructions before changing request content to preserve a reusable prefix. Measure actual cache usage rather than assuming every repeated request receives cached pricing.

Verdict: Caching and trimming are complementary. Keep valuable stable instructions cache-friendly, but do not preserve dead text merely because it might be cached.

The winning prompt survives the same incident queue

Run both prompts against representative tasks, malformed inputs, ambiguous requests, and attempts to override instructions. Use the same model configuration, tools, and scoring criteria. Repeat tests where output variability matters. Record billed input categories, output tokens, latency, tool calls, retries, and task success.

If comparing Claude Haiku 4.5 with Gemini 2.5 Flash, evaluate each prompt within each model first. Otherwise, a model change can obscure whether trimming helped. Follow the offline test with a limited rollout and a rollback path; average scores can hide a rare but serious regression.

Verdict: Choose the smallest prompt that preserves required behavior across the evaluation set, not the shortest prompt that passes a demonstration.

The handover decision: trim first, then compare routes

Relay365 fits when the next step is testing across model families through a pay-as-you-go relay desk. It works with Claude Code, Codex, Gemini CLI, and OpenAI SDK clients. Check route-specific pricing, caching behavior, and feature compatibility before drawing cost conclusions. Create a Relay365 account to run a controlled comparison on your own workload.

The final call is operational: retain essential constraints, move changing knowledge out of the permanent prefix, measure cold and warm traffic separately, and reject savings that increase failures. A system prompt should be a maintained contract, not an archive of every incident.