Duty log: model API outage review ready for handoff

Open the review while the outage is still fresh

A model API outage is any disruption that prevents your application from getting usable responses within its expected limits. It can mean failed requests, unusually slow replies, interrupted streams, or access failures affecting one model. You do not need a complete service shutdown to write an incident review.

The review is a shared record of what happened, who was affected, and what needs to change. It is not a search for someone to blame. Start with available evidence, mark unknowns clearly, and update the document when better information arrives.

Record the incident boundaries before explaining the cause

Copy these prompts into your incident document and replace the bracketed text. Keep the summary understandable to someone who was not on duty.

• The incident was tracked as [incident ID], with [owner] responsible for completing this review.
• The affected application was [application], using [model], through [direct provider or relay], in [environment].
• The first observed failure occurred at [timestamp and time zone], and service recovery was confirmed at [timestamp and time zone].
• Users experienced [visible symptom], affecting [measured scope or an explicit statement that scope is unknown].
• The current incident state is [ongoing, mitigated, or resolved], with [remaining limitations].

Distinguish the first observed failure from the actual start of the outage. If monitoring cannot establish the start time, say so. Do not turn an estimate into a precise timestamp.

Build a timeline from requests and decisions

Use one time zone throughout. Capture the sequence of symptoms, alerts, investigations, changes, and recovery checks. Include decisions that did not work; they explain the response and can improve future runbooks.

• At [time], [monitor or person] detected [symptom], supported by [log reference or metric].
• At [time], the team tested [hypothesis] by [check], which showed [result].
• At [time], [responder] applied [change], and observed [effect].
• At [time], recovery checks confirmed [successful behavior] across [relevant request types].

Save request IDs, status codes, latency measurements, and sanitized error messages. For streaming requests, record whether failure happened before output or midway through a response. Remove API keys, personal information, and sensitive prompt content before sharing evidence.

Separate the failure location from the suspected cause

A failed model request can originate in your application, network, relay, authentication setup, account limits, or upstream model service. An error code is evidence, not a complete diagnosis. A timeout alone does not prove that the model provider was unavailable.

• The confirmed failure was [observed behavior], supported by [evidence].
• The likely contributing factors were [factors], with [evidence or uncertainty] recorded for each.
• The checks that ruled out alternatives were [checks and results].
• The unanswered questions are [questions], assigned to [owner] for follow-up.

For example, if requests to Claude Haiku 4.5 failed while another configured model worked, record that narrower scope. Do not describe every model or the entire API as unavailable without supporting evidence.

Document recovery without hiding its trade-offs

Explain whether you restored the original path, reduced traffic, disabled a feature, or switched models. A fallback may change output quality, tool behavior, latency, or cost, so successful HTTP responses alone are not enough to confirm recovery.

Record retry limits, backoff, and request timeouts. Repeating requests can increase load and cost, and repeating an agent workflow can duplicate downstream actions. Note how you checked for those effects.

Leave the next shift with owned follow-up work

Each action should name an owner, a target date, and a verification step. Prefer “test interrupted-stream handling in staging” over “improve reliability.” Include any monitoring gaps that delayed detection.

If the incident exposed a need to evaluate another access route, Relay365 is a pay-as-you-go relay desk for multiple model families. Treat it as a dependency to test, not a guarantee against upstream outages. Confirm model compatibility, error visibility, and costs before changing production routing. Create a Relay365 account if that evaluation fits your follow-up plan.

Close the review when the evidence, remaining uncertainty, and next actions are clear enough for another engineer to take over.