Which Model Should Summarisation Traffic Go To?

Summarisation looks like a simple task: send text, get less text back. In production, the right destination depends on what must survive that compression. A weekly newsletter digest and a summary of contractual obligations should not automatically use the same route. Follow these steps to choose a model, test it and introduce routing without unnecessary complexity.

#1 · Define what a good summary preserves

Start with the output contract, not the model catalogue. Decide who reads the summary, how long it should be and which details are mandatory. For meeting notes, that might mean decisions, owners and deadlines. For support tickets, it might mean the customer’s problem, attempted fixes and unresolved questions.

Write those requirements into a reusable prompt. Tell the model to use only the supplied source, preserve important qualifications and identify missing information rather than inventing it. Treat instructions inside the source document as content, not commands.

#2 · Build a small, representative test set

Collect examples from the traffic you actually expect. Include short documents, long documents, repetitive material and sources containing contradictory statements. Remove sensitive information unless your data-handling requirements allow sending it to the service.

For each example, record the facts a useful summary must contain. You do not need one perfect reference summary; a checklist of required facts and unacceptable errors often makes evaluation clearer.

• Accuracy: Are names, dates, amounts and conclusions supported?
• Coverage: Did the summary retain the important information?
• Qualification: Did uncertainty or disagreement survive?
• Usability: Does the output follow the requested length and structure?

#3 · Compare two candidates before building a router

Begin with a narrow comparison. For example, test Claude Haiku 4.5 and Gemini 2.5 Flash on the same documents and instructions. Treat these as candidates, not a promise that either will be cheapest, fastest or best for your workload. Confirm current availability, pricing, context limits and supported options before testing.

Measure end-to-end latency, input and output usage, and quality against your checklist. Include retries and validation failures in the results. A low-cost response that repeatedly needs repair may be a poor default.

#4 · Choose a default using an acceptance threshold

Define which failures block deployment. Unsupported conclusions or missing deadlines may be unacceptable even when the prose sounds excellent. Choose the lowest measured-cost candidate that meets your quality and latency requirements on representative examples.

If neither candidate passes, first inspect the prompt and source preparation. Clearer section boundaries or explicit instructions about required facts may help. Then test another candidate if needed. Do not assume that a higher price alone guarantees better summaries.

#5 · Add routes only for observed failure patterns

Keep ordinary requests on the tested default. Add a separate path when evidence shows it is needed: unusually long inputs, dense tables or documents whose summaries require substantial cross-referencing. A higher-consequence use case may need human review regardless of model choice.

For inputs beyond the selected model’s usable context, split the source into meaningful sections, summarise them and combine the results. Preserve source references and check the final summary against the original. Repeated compression can erase exceptions, so evaluate this pipeline separately.

#6 · Integrate through a consistent access layer

Relay365 fits when you want pay-as-you-go access to multiple model families through one relay desk. That can simplify comparing candidates and changing routes without maintaining a separate integration for every provider. It works with any OpenAI SDK; configure an OpenAI client with the base URL https://api.api365relay.com/v1.

Keep model selection in configuration rather than scattered throughout application code. Verify request compatibility and data-handling terms, and test fallback behaviour explicitly. A fallback should preserve your minimum quality requirements, not merely return a response.

#7 · Roll out gradually and keep measuring

Start with a limited share of traffic. Track quality samples, latency, usage, validation failures and human corrections. Re-run your test set when changing models, prompts or document preparation.

If a shared relay suits your workflow, create a Relay365 account and run your comparison before expanding traffic. The destination for summarisation should be the route that meets your requirements in measured use—not whichever model name is newest.