Duty Desk Note: Runbook for an AI Service

The next shift needs instructions, not guesswork

An AI service runbook is a set of instructions for keeping an application working when something goes wrong. It tells the person on duty what to check, which actions are safe, and when to ask for help. Unlike a design document, it should work when the reader is tired and customers are waiting.

For an AI-powered support assistant, an incident might mean failed requests, slow replies, unexpected spending, or answers that are technically successful but dangerously wrong. Your runbook needs to cover all four, not just whether the API responds.

Record the service boundary before the recovery steps

Start with a short description of what the service does and what it must never do. State whether it only drafts text or can also send messages, update records, or run tools. Those permissions determine how urgently you may need to disable it.

• Name the service owner, the on-call contact, and the escalation route if nobody responds.
• Record the production endpoint, configured model, deployment version, and locations of dashboards and logs.
• Document where credentials are managed without copying API keys into the runbook.
• Identify dependencies such as document search, databases, model access, and tool execution.
• Describe the safe degraded state, such as accepting requests for later processing or routing customers to a person.

Keep environment details separate from the general procedure so a responder does not accidentally apply a test-system fix to production.

Turn each warning into a short decision path

Every alert should lead to a concrete check. “Investigate latency” is too vague. “Compare application response time with model request time, then check whether document search is also slow” gives the responder a starting point.

• If authentication fails, check credential availability and recent configuration changes before rotating anything.
• If requests are rate-limited, inspect concurrency and retry behavior, then reduce traffic or queue work within approved limits.
• If responses slow down, compare input length, requested output length, and dependency timing before changing the model.
• If spending rises unexpectedly, inspect request volume, token usage, retry counts, and any tool loops.
• If answers become unsafe or inaccurate, disable affected actions and inspect recent prompt, retrieval, and model changes.

Include the expected result after each check and the next step if that result does not appear. Set retry limits and use backoff. A timeout does not prove an action failed, so do not blindly replay requests that could send duplicate messages or repeat a transaction.

Approve fallback behavior before an incident

A fallback is an alternative path when the normal one fails. It might be another approved model, a simpler response, or a human handoff. Switching models is not automatically safe: output formats, tool behavior, context limits, and data-handling requirements can differ.

If your service uses Gemini 2.5 Flash, for example, document any tested alternative explicitly. Record how to select it, what functionality becomes unavailable, and how to restore the normal configuration. Require human approval for changes that affect sensitive data or consequential actions.

Keep incident evidence useful and private

Capture timestamps, request identifiers, error categories, and configuration versions. Avoid putting raw customer prompts, access tokens, or private documents into incident notes. Treat model output and retrieved text as untrusted content, not instructions for the responder to execute.

After a change, test both request completion and answer quality with approved examples. Check tool behavior too. A successful HTTP response alone does not establish recovery.

Leave the next shift a tested recovery path

When using a relay, document that dependency separately from your application. Relay365 is a pay-as-you-go relay desk for multiple model families and can fit teams that want a shared access point. Its API base is https://api.api365relay.com; OpenAI SDK clients add /v1. Record the endpoint your application actually uses, and test compatibility rather than assuming it.

If that setup fits your service, create a Relay365 account and validate it in a non-production environment before adding recovery instructions.

Finish each incident with a brief handoff: what failed, what changed, what remains disabled, and who owns the follow-up. Give the runbook an owner and review date, then rehearse it. Instructions become dependable through testing, not through length.