Shift Note: Internal Docs Chatbot Entering Pilot

The first job is finding the right internal page

An internal documentation chatbot lets employees ask questions such as “Where do I request production access?” and receive an answer grounded in company documents. Instead of guessing which folder contains the instructions, they describe what they need in ordinary language.

The chatbot should be a reading assistant, not a new source of company policy. Its job is to find relevant material, explain it clearly, and identify the sources. If the documentation does not answer the question, saying so is more useful than inventing a plausible procedure.

Start with one collection and one recurring question type

Choose a narrow starting point, such as onboarding instructions or service runbooks. Avoid connecting every shared drive on day one. A smaller collection makes incorrect answers easier to investigate and gives document owners a manageable review task.

• Select documents with clear owners and reasonably current instructions.
• Record each document’s title, location, last-updated date, and access permissions.
• Collect real questions that employees already ask in support channels.
• Decide who will correct missing, outdated, or contradictory documentation.

This preparation matters because a chatbot cannot reliably repair a confused knowledge base. If two pages disagree about the escalation process, the underlying disagreement needs an owner.

Give the model relevant excerpts before it answers

A common approach is retrieval-augmented generation, usually shortened to RAG. In plain language, the application searches your documents first and then asks a language model to answer using the passages it found. You generally do not need to train a model on your entire documentation collection.

The preparation step divides documents into smaller passages while preserving useful context, such as section headings. A search index stores those passages. Some systems also use embeddings, which are numerical representations that help match a question with passages about similar ideas, even when the wording differs.

When someone asks a question, the application retrieves relevant passages and sends them, together with the question, to the model. The instructions should require an answer supported by those passages, source references, and a clear admission when evidence is missing. Your application should render source references from known document metadata rather than trust invented locations.

Check permissions before passages reach the model

Access control belongs in the application, not just in the prompt. Identify the employee and filter retrieval results using their existing document permissions before sending any excerpts to the model. A sentence telling the model not to reveal confidential information is not an authorization system.

Treat document text as untrusted input, too. A page might contain instructions that try to redirect the chatbot or request secrets. Keep retrieved content separate from application instructions, and do not give a documentation assistant unnecessary tools or write access.

Review provider data-handling terms, retention settings, logging, and organizational approval before sending internal content to any external service. These checks also apply when requests pass through an intermediary.

Test answers before opening the support channel

Run a small pilot with representative questions, including questions with no documented answer. Check whether the search found the right material, whether the response accurately reflects it, and whether the employee can access every cited source.

• Test restricted documents with accounts that have different permissions.
• Include stale pages and conflicting instructions to check whether uncertainty is surfaced.
• Measure response time and request cost using your own workload.
• Add a feedback route that captures the question and source references without unnecessarily logging sensitive content.

Choose the model connection after the guardrails

Relay365 can fit when you want a pay-as-you-go relay desk for multiple model families through a common connection. It works with OpenAI SDK clients; its API base is https://api.api365relay.com, with /v1 added for those clients. You could evaluate Gemini 2.5 Flash against your pilot questions rather than choose on name alone.

If that setup matches your approved architecture and data-handling requirements, create a Relay365 account to try the pilot. Keep credentials on the server, set usage limits where supported, and expand the document collection only after the first group can consistently find trustworthy answers.