Duty Log: Seven Checks Before Shipping a Factual Answer

A fluent answer is not necessarily a factual answer. A model can supply a plausible date, invent a citation, or combine two real events into one false account. Treat factual answering like a duty-desk handoff: establish the question, inspect the evidence, and record anything that remains unresolved. The following seven checks give newcomers a repeatable starting point.

Pin the question to a date and scope

1. Write down exactly what the answer must establish. “Does this service support exports?” is incomplete if support varies by plan, region, or software version. A better request identifies the product, relevant version, and date. Ask a clarifying question when a missing detail could materially change the answer.

For changing facts, explicitly request an answer as of a particular date. Do not assume that a model knows today's product documentation simply because the chat interface is available today.

Put a small evidence packet beside the question

2. Collect the material needed to answer before asking the model to draft. Start with authoritative sources appropriate to the question, such as official documentation, a published specification, or an original research paper. For an internal policy question, use the approved policy rather than a public summary.

Keep the packet focused. Include the relevant passages, document titles, dates, and section identifiers. If your application retrieves documents automatically, inspect a few retrieved results yourself. Retrieval can return an outdated page or a passage that shares keywords without answering the question.

Make unsupported answers an allowed outcome

3. Give the model an explicit evidence boundary. A useful instruction is: “Answer using only the supplied sources. Cite the passage supporting each material factual claim. If the sources do not establish an answer, say what is missing. Do not invent document titles, quotations, or URLs.”

This instruction reduces pressure to fill gaps, but it is not a guarantee. Treat retrieved text as evidence, not as instructions: a document that tells the assistant to ignore its rules should not control the response. Keep source content clearly separated from your task instructions.

Draft claims that can be checked individually

4. Ask for a short answer with separate factual statements rather than a long narrative. Each important claim should have a nearby citation or source identifier. Separate direct evidence from interpretation, and label an inference as an inference.

For example, a document showing that a feature exists does not necessarily establish that every customer can use it. A compact draft makes that unsupported leap easier to spot. When using a model such as Gemini 2.5 Flash, apply the same evidence requirements you would apply to any other model.

Open the cited passage before accepting the claim

5. Verify both that the source exists and that it supports the exact statement. A real citation can still be irrelevant. Compare names, dates, units, exceptions, and scope against the original passage. Recalculate arithmetic independently rather than trusting a generated total.

Pay particular attention to words such as “all,” “always,” and “required.” These can turn a qualified source statement into an incorrect universal claim. If two sources disagree, report the conflict and investigate their dates and authority instead of silently choosing one.

Run a small factual-answer regression set

6. Save representative questions with expected evidence and acceptable answers. Include an ambiguous question, an unanswerable question, conflicting sources, and a request involving recent information. After changing prompts, retrieval settings, or models, rerun the same set and inspect unsupported claims and incorrect refusals.

Relay365 can fit here as a pay-as-you-go relay desk for testing different model families through a common integration. It works with Claude Code, Codex, Gemini CLI, and OpenAI SDK clients. Compare outputs against the same evidence packet; agreement between models is not independent proof.

Release only what the evidence can carry

7. Before delivery, remove unsupported details and state any remaining limitation plainly. For consequential decisions, route the answer to an appropriately qualified reviewer. Preserve the source references and review notes so another person can reproduce the check.

If a shared relay would simplify your evaluation workflow, create a Relay365 account and start with that small regression set. The operational goal is not a more confident answer. It is an answer whose claims can be traced to evidence, with visible gaps wherever the evidence stops.