Shift Note: Temperature Settings Before the Next Run

The handoff starts with output variation

The prompt stayed the same, but the answer changed. That is often normal behavior, not a broken integration. Language models generate text in small pieces called tokens. At each step, the model assigns probabilities to possible next tokens, and a selection process chooses one.

Sampling settings influence that selection. They can make outputs more conservative or more varied, but they do not turn factual accuracy on or off. For a new application, the useful question is not which temperature is best. It is how much variation the job can tolerate.

Temperature changes how strongly the favorite wins

Temperature adjusts the balance among likely next tokens. A lower temperature makes high-probability choices more dominant. A higher temperature gives less likely choices more opportunity to appear.

Imagine a sentence beginning, “Please restart the…” A low setting tends to favor familiar continuations. A higher setting allows a wider range. Across a whole response, that can produce different phrasing, examples, and directions.

Higher temperature is not a creativity guarantee, and lower temperature is not a truth guarantee. A low-temperature answer can repeat a confident mistake. A high-temperature answer can still be useful and accurate. Instructions, context, and the model itself remain important.

Top-p and top-k narrow the candidate pool

Top-p, also called nucleus sampling, limits selection to a group of tokens whose combined probability reaches a chosen threshold. With top-p set to 0.9, the sampler generally keeps the smallest group covering at least 90% of the probability mass. That does not mean the response is 90% accurate.

Top-k instead limits selection to a fixed number of the most likely tokens. A top-k of 40 considers up to the 40 highest-ranked candidates at each step. Not every API exposes this setting.

Temperature reshapes probabilities; top-p and top-k restrict candidates. Their interaction and processing order can depend on the implementation. For a beginner, changing all three at once makes troubleshooting unnecessarily difficult.

Choose a starting point for the actual task

Check the selected model’s documentation before setting values. Accepted ranges, defaults, and restrictions differ. Some models or modes do not allow temperature adjustments, and an unsupported parameter may be rejected or ignored.

• For extraction, classification, or routine transformations, start with a low temperature, such as 0 to 0.3, if supported, and validate the result against explicit requirements.
• For general explanations or customer-facing drafts, keep the documented default first, then lower temperature if variation becomes distracting.
• For brainstorming, try a moderately higher temperature, such as 0.7 to 1.0 where supported, and evaluate whether the alternatives are genuinely useful.
• Leave top-p at its documented default while testing temperature, then evaluate top-p separately if you need more control.

These are trial settings, not portable rules. A value of 0.7 on Gemini 2.5 Flash should not be assumed to behave identically on another model.

Temperature zero does not lock the answer

At temperature zero, implementations typically favor the highest-probability next token. Even then, identical requests are not guaranteed to produce identical responses. Backend execution, model updates, and other implementation details can affect results. A seed, when supported, may improve repeatability without guaranteeing it.

Keep a small evaluation set containing ordinary requests and awkward edge cases. Run each candidate setting several times. Record the model identifier, prompt, parameters, output limits, and results. Check correctness, format compliance, and variation separately. Use schema validation or application checks rather than relying on temperature to enforce structure.

Keep the relay convenient and the settings explicit

If you want to test across model families, Relay365 is a pay-as-you-go relay desk that can fit that workflow. It works with Claude Code, Codex, Gemini CLI, and OpenAI SDK clients. A shared access layer is convenient, but it does not make every model’s sampling controls interchangeable.

Create a Relay365 account if that setup fits your workflow, then verify parameter support for each selected model. The useful shift handoff is simple: save the settings, keep the test cases, and promote a configuration only when its outputs meet the job’s requirements.