Duty log: one handoff prompt, three different answers

The handoff prompt stayed put; the wording moved

You ask an assistant to turn an incident note into a shift handoff. You send the same prompt three times. One answer leads with customer impact, another leads with the investigation, and the third puts the next action first.

That does not automatically mean the service is broken or your prompt changed. Many language models generate answers using sampling: a process that allows more than one plausible continuation instead of always choosing the single highest-ranked next piece of text.

For a duty desk, the useful question is not whether every word matches. It is whether every answer preserves the facts, respects uncertainty, and gives the incoming operator a safe handoff.

Each next token offers another small fork

A model builds its response in tokens, which are pieces of words, punctuation, or other text. At each step, it assigns probabilities to possible next tokens. Sampling chooses from that distribution, subject to the settings supported by the model and endpoint.

Think of an operator choosing between “Investigating elevated errors” and “Elevated errors remain under investigation.” Both can express the same state. Once the first few tokens differ, later choices happen in a different context, so the answers can drift further apart.

This is not the model searching a drawer for three finished replies. It is constructing a reply step by step. Variation can be harmless wording, but it can also affect emphasis, completeness, or factual accuracy.

Three handoffs can vary without changing the incident

Suppose the source note says: “Errors increased after the deployment. Rollback is complete. Recovery is not yet confirmed. Continue monitoring.” These are illustrative outputs, not recorded model results:

• One answer might say, “Rollback is complete; continue monitoring because recovery has not been confirmed.”
• Another might say, “Errors increased after deployment, and the team has completed a rollback. Recovery remains unconfirmed.”
• A third might say, “The next shift should monitor for recovery following the completed rollback.”

The first two explicitly preserve the unresolved status. The third leaves it implicit and drops the original error increase. A reviewer might reject it even though it sounds reasonable. Fluent wording is not evidence that a handoff is complete.

Temperature narrows choices, not operational responsibility

Where supported, temperature changes how strongly the model favors higher-probability tokens. Lower values generally make choices more concentrated; higher values generally allow more variation. Temperature is not an accuracy dial, and a low setting can still produce a confident mistake.

Another common control, top-p, limits sampling to a set of tokens whose combined probability reaches a chosen threshold. For early experiments, adjust one control at a time. Changing both makes it harder to tell which adjustment mattered.

Support, accepted ranges, and defaults vary by model and endpoint. Some expose fewer controls. Even temperature zero should not be treated as a promise of identical output: serving infrastructure, model revisions, and numerical execution can introduce differences. A seed, when available, is a reproducibility aid rather than a universal guarantee.

The shift test should score facts before phrasing

Before testing, define the required facts and forbidden claims. For this incident, every handoff should preserve the completed rollback, unconfirmed recovery, and instruction to monitor. None should announce resolution.

• Keep the complete request fixed, including system instructions, conversation history, tool results, and generation settings.
• Run it several times and check each answer against the same factual checklist.
• Record the requested model identifier, settings, timestamp, and any returned version information available.
• Require human review before an answer changes incident status or triggers an operational action.

A relay desk helps organize the next comparison

If your team needs access to multiple model families, Relay365 is a pay-as-you-go relay desk worth considering. You could repeat this handoff test with Claude Haiku 4.5 and Gemini 2.5 Flash, checking each endpoint’s supported controls rather than assuming identical behavior.

Relay365 works with Claude Code, Codex, Gemini CLI, and OpenAI SDK clients. Its API base is https://api.api365relay.com; OpenAI clients add /v1. Create an account when you are ready to run a small, logged evaluation.

The shift-note rule stays simple: allow variation in expression, test for consistency in facts, and never let a polished summary silently turn “not confirmed” into “resolved.”