Find Expensive Requests in Your Traffic Log

A high API bill rarely tells you which application behavior caused it. Your traffic log can help: each request connects a model choice, a workload, and a cost. The goal is not simply to find the biggest number. It is to identify spending that produces too little value. Follow this walkthrough using your gateway logs, application telemetry, or an exported usage report.

#1 · Choose a small, complete time window

Start with one full day or a recent development session. Use a consistent time zone and include all relevant applications or API keys. A narrow window is easier to investigate than a month of mixed traffic.

Compare the log's totals with your billing or usage report. Differences can come from reporting delays, missing records, or different billing boundaries. Treat your first export as an investigation dataset, not automatically as an exact invoice.

#2 · Collect the fields that explain spending

Use whatever fields your logging system provides. If something is missing, add it to future application telemetry rather than guessing.

• Identity: timestamp, request ID, application, environment, and a pseudonymous user or job identifier.
• Routing: requested model and, when available, the model actually used.
• Usage: input tokens, output tokens, and separately reported cache or other billable usage categories.
• Outcome: status, latency, retry attempt, and billed cost when available.

Do not export API keys or unnecessarily copy prompt contents. Request metadata is often enough to locate the responsible code path.

#3 · Sort by cost, not just token count

If the log includes a documented per-request billed amount, sort it from highest to lowest. Otherwise, estimate cost using the applicable model rates and billing categories for the period you are investigating.

For a basic token-priced request, multiply input tokens by the input rate and output tokens by the output rate, then add the results. Match the rate's unit: a price per million tokens requires dividing token counts by one million. Account for cache pricing or other charges only when applicable, and avoid counting the same usage twice.

Tokens alone are not a reliable cost ranking. Requests using Claude Haiku 4.5 and Claude Opus 5.5 can have different prices even when their token counts match.

#4 · Group requests to find the real hotspots

Inspect the largest individual requests, then group spending by endpoint, task, model, environment, and user or job. Calculate total cost, request count, and average cost per request for each group.

This reveals two different problems: one unusually expensive operation, or thousands of ordinary operations that add up. A frequently refreshed summary can cost more overall than an occasional long analysis. Keep both the individual-request ranking and the grouped totals.

#5 · Trace expensive requests back to their cause

Open a few representative records from each hotspot. Look for repeated conversation history, oversized retrieved documents, verbose outputs, or a more capable model being used for a straightforward task.

Check retries especially carefully. Several attempts may belong to one user action, so use a shared operation identifier when possible. A timeout does not prove that no billable processing occurred; verify recorded usage and the service's billing behavior. Also inspect agent loops that keep making requests without reaching a useful stopping point.

#6 · Make one controlled change

Choose the clearest source of waste first. Trim irrelevant context, retrieve fewer but better passages, set a suitable output limit, or stop redundant retries. For simpler tasks, test a less expensive model after checking current rates.

Replay a small, representative set of tasks and compare cost alongside answer quality, latency, and failure rate. Lower spending is not an improvement if users must repeat the task. Keep the original configuration available while evaluating the change.

#7 · Repeat the review with consistent attribution

Save your grouping method and review the same metrics regularly. Label new features and background jobs so future spending has an identifiable owner.

If your application uses several model families, Relay365 can fit as a pay-as-you-go relay desk for consolidating access. It works with Claude Code, Codex, Gemini CLI, and OpenAI SDK clients. Create a Relay365 account if that setup matches your workflow, and confirm which usage fields are available before relying on any report for request-level analysis. Consistent application-side identifiers remain useful regardless of your provider.