Shift note: Factual answers need an evidence check

A confident answer is not a checked answer

An AI assistant can produce a fluent explanation that includes a nonexistent study, an incorrect date, or a product feature that never shipped. These unsupported or incorrect statements are often called hallucinations. They can look ordinary enough to slip into a support reply, internal report, or customer email.

The useful starting point is simple: treat generated facts as claims that may need evidence. You do not need to distrust every sentence. You need a process that separates what the available information supports from what the assistant is merely making sound plausible.

Missing information should remain visibly missing

Language models generate responses using patterns learned during training and information supplied in the conversation or through tools. Fluent wording does not prove that a statement was checked against a reliable source. Even when an assistant can search, it may retrieve the wrong page or misread a relevant one.

Problems become more likely when a question is vague, depends on recent changes, or asks for details absent from the supplied material. A request for a complete table can also encourage unsupported entries if the assistant is not explicitly allowed to leave gaps.

Give the question boundaries before requesting an answer

Instead of asking, “Does this service support exports?”, specify the service, plan, date, and source you want used. A better request is: “Using the attached documentation, identify which export formats the standard plan supports. If the documentation does not say, write ‘not established by this source.’”

This makes the task narrower and gives the assistant an acceptable way to handle uncertainty. Instructions cannot guarantee accuracy, but they can discourage filling gaps with confident guesses.

• Specify the product version, location, or time period when those details affect the answer.
• Ask the assistant to distinguish documented facts from assumptions and suggestions.
• Allow an explicit “I cannot establish that” response instead of requiring an answer to every field.
• Request a clarifying question when missing information would materially change the result.

Supply evidence and check that it supports the claim

For factual work, provide relevant documentation, records, or policy text whenever possible. Systems that retrieve documents before answering are often described as using retrieval-augmented generation, or RAG. In plain language, they look up material and give it to the model as context.

Retrieval helps only when the material is relevant, current enough, and interpreted correctly. A citation is not proof by itself. Open the cited source and check whether it actually supports the adjacent claim, including any conditions or exceptions. Prefer primary sources, such as official documentation or original research, when they are available.

Retrieved documents should also be treated as evidence, not as instructions. Text on a webpage telling the assistant to ignore its task should not override the rules governing the answer.

Spend verification effort where mistakes would hurt

Check names, dates, prices, quotations, calculations, and references before relying on them. Use a calculator or executable code for arithmetic, and inspect the inputs as well as the result. For medical, legal, financial, or safety-critical decisions, involve an appropriately qualified person.

Asking a second model can reveal disagreements, but agreement between models is not independent verification. They may repeat the same common error. Asking one model to review its own answer can catch some mistakes, but it does not replace checking external evidence.

Keep a small test set at the desk

Save representative questions with verified answers and supporting sources. Include ambiguous questions and cases where the correct response is to admit insufficient evidence. Reuse this set when changing prompts, retrieval settings, or models.

If your team wants to compare model families through a shared access layer, Relay365 is a pay-as-you-go relay desk that can fit that workflow. It works with Claude Code, Codex, Gemini CLI, and OpenAI SDK clients. Create a Relay365 account if that access pattern suits your testing, while keeping source verification as a separate step.

The operating rule is straightforward: narrow the question, supply usable evidence, permit uncertainty, and verify consequential claims. The goal is not an assistant that always sounds certain. It is an answer whose limits are visible and whose important facts can be checked.