Release Desk: Seven Checks Before a Prompt Goes Live

A prompt that worked in one chat is not ready for production. Before releasing it, collect evidence that it handles normal requests, awkward inputs, and predictable failures. Follow these seven checks in order, and keep the results beside the prompt version so the next person on duty can understand the release decision.

Write Down the Prompt’s Release Contract

1. State the job in one sentence, then define what a correct response must contain. Include the intended audience, required format, allowed information sources, and actions the assistant must not take. Replace vague requirements such as “be helpful” with observable behavior, such as “ask for the missing order number before requesting a refund lookup.”

Choose release blockers before testing. For example, malformed JSON or an invented refund confirmation should block release. A slightly repetitive sentence might be a lower-priority issue. This prevents a polished answer from distracting you from a broken requirement.

Build a Small Set of Representative Inputs

2. Create a test file with ordinary requests, incomplete requests, and difficult requests. Include at least one example of every important rule in the contract. Use synthetic or appropriately sanitized data, especially when testing customer support, financial information, or internal documents.

• Include a straightforward request with enough information to complete the task.
• Include a request with a missing field that should trigger a clarification.
• Include an ambiguous request with more than one reasonable interpretation.
• Include an unsupported request that should receive a clear boundary or handoff.

Write the expected behavior beside each input. You do not need a perfect reference answer when several phrasings would be acceptable.

Capture a Baseline With Fixed Settings

3. Run the current prompt against the full test set before editing it. Record the prompt text, model identifier, generation settings, tool definitions, and relevant application configuration. Keep the raw responses, not just your assessment of them.

Repeat important cases several times because generated responses can vary. A single successful run does not establish consistent behavior. If you change the model and prompt together, record both changes; otherwise, it becomes difficult to identify which change caused a regression.

Probe Instruction Conflicts and Untrusted Content

4. Add inputs that ask the assistant to ignore its instructions, reveal protected information, or follow commands embedded in retrieved documents. Put those commands inside content the application normally processes, such as a support ticket or search result.

Check whether the assistant treats that material as data rather than authority. Also test requests that imply permissions the user has not established. Prompt wording is not an access-control mechanism, so verify that the application independently enforces permissions before any sensitive tool action.

Validate Outputs Where the Application Uses Them

5. Feed responses into the same parser, schema validator, or rendering path used in production. A response that looks like JSON can still fail parsing, omit a required field, or contain an invalid value. For tool calls, inspect the tool name and arguments before allowing execution.

Record observed latency and token usage during these runs. Compare them with your application’s own limits, including timeout and retry behavior. Do not infer production cost or responsiveness from one short example.

Rerun the Checklist on Every Intended Model

6. Test each model your application will actually route to, including any fallback. For example, a prompt evaluated on Claude Haiku 4.5 still needs its own run on Gemini 2.5 Flash if both are release targets. Shared API conventions do not imply identical instruction handling or tool behavior.

Relay365 is a practical option when this work spans multiple model families: its pay-as-you-go relay desk supports tools including Claude Code, Codex, and Gemini CLI, as well as OpenAI SDK clients. Keep model-specific results separate even when access runs through one service.

Record the Release Decision and Rollback Target

7. Rerun the entire test set after the final edit. Record unresolved failures, the person approving release, and the previous prompt version to restore. Start with limited exposure where your application supports it, and identify which production signals will trigger investigation or rollback.

If Relay365 fits your testing workflow, create an account to run your model checks through the relay. Release only when the recorded evidence meets your acceptance criteria, and retain the checklist for the next prompt change.