Build a Chatbot for Your Internal Documentation

Define one useful documentation question

1. Start with a narrow job, such as answering onboarding questions from your engineering handbook. Avoid connecting every company document immediately. Choose a small collection with a clear owner, and write down questions employees actually ask. Define success in observable terms: the chatbot finds the right policy, explains it accurately, and points to the source. Include questions it should decline because the documentation does not contain an answer.

Prepare documents with ownership and permissions

2. Gather the approved documents and remove obsolete copies, broken exports, and navigation clutter. Preserve headings, numbered instructions, and code examples because their structure helps explain meaning. Give each document a stable identifier, title, source location, owner, and update timestamp. Record who may read it. Start with material available to your pilot audience, but design the system so restricted documents cannot accidentally become searchable later.

Build a searchable collection of document passages

3. Split documents into passages that retain enough context to stand alone. Prefer section boundaries over arbitrary cuts, and keep each passage connected to its parent document and permissions. Create a search index using keyword search, embeddings, or both. Embeddings represent text numerically for similarity search; they do not replace access controls. Inspect results for your sample questions before adding a language model. Poor retrieval is easier to diagnose without generated prose hiding the problem.

Connect a model through a replaceable interface

4. Put model access behind a small server-side function rather than calling it directly from the browser. Keep credentials in your server environment or secret manager. Relay365 can fit here as a pay-as-you-go relay desk for multiple model families, particularly if you want an OpenAI SDK-compatible integration. Use a configurable model setting, with Gemini 2.5 Flash as one possible example, and confirm the exact identifier and supported features before implementation. Configure document indexing separately rather than assuming the same endpoint provides embeddings.

Generate answers from retrieved evidence

5. For each question, authenticate the employee, retrieve passages they are authorized to read, and send only the relevant material to the model. This approach is commonly called retrieval-augmented generation, or RAG. Tell the model to answer from the supplied evidence, cite passage identifiers, and acknowledge missing information. Validate returned identifiers against the retrieved set before displaying source references. Keep conversation history bounded, and do not silently treat an earlier generated answer as documentary evidence.

Protect the boundary between documents and instructions

6. Treat retrieved text as untrusted content, even when it comes from an internal repository. A document might contain instructions that attempt to redirect the chatbot. Tell the model that passages are evidence, not commands, and enforce permissions in application code rather than in a prompt. For a first release, keep the chatbot read-only. Do not give it deployment, deletion, or account-management capabilities merely because a document describes those operations.

Test answers against a small reference set

7. Run the same questions after every meaningful change. Include straightforward lookups, ambiguous requests, outdated procedures, and attempts to access restricted information. Test with separate accounts representing different permission levels. Review retrieval and generation independently so you know which component needs attention.

• Check that each factual answer is supported by the passages shown to the model.
• Check that source references resolve to the correct document and section.
• Check that unanswered questions produce an honest limitation rather than an invented procedure.
• Check that restricted material never appears in retrieved context, answers, or exposed logs.

Launch a limited pilot and maintain the index

8. Release to a small group, provide a feedback button, and assign someone to review incorrect answers. Track request cost and response time without inventing performance targets. Refresh changed documents and remove deleted content from the index. Recheck authorization when users follow source references, and set retention rules for conversations and logs. If Relay365 suits your integration, create an account and test it with non-sensitive sample documents first. Expand only after retrieval quality, permission checks, and failure handling behave as expected.