Handle 429 Rate Limits Without Retry Storms

Recognize what a 429 response is telling you

1. Treat HTTP 429 as a signal to slow down and inspect the response. It commonly means your application has exceeded a request, token, or concurrency limit. Some services also use it for exhausted quota or account restrictions. An immediate retry can make temporary congestion worse, while repeated retries cannot fix a depleted balance.

Read the response body and relevant headers before deciding what happens next. Record the error code, request identifier, model, and time. Keep API keys and sensitive prompt content out of logs. Your first goal is to distinguish a temporary traffic limit from a condition that requires an account or configuration change.

Honor the server’s suggested retry time

2. Look for the Retry-After header. Its value can be a number of seconds or an HTTP date. If it says 12 seconds, wait at least that long before trying again. For a date, calculate the remaining time using a reliable system clock.

Not every API sends this header. Some expose additional rate limit headers, but their names and meanings vary. Follow the provider’s documentation instead of assuming every service uses the same format. If the suggested wait exceeds your application’s deadline, stop the automatic retry and return a useful status to the caller.

Use exponential backoff with random jitter

3. When the response is retryable and provides no usable waiting instruction, increase the delay between attempts. A simple starting policy is a one-second base delay, a 30-second cap, and no more than four retries after the initial request. These are example settings to tune for your application.

Use exponential backoff to grow the delay, then add randomness, called jitter. With full jitter, choose a random delay between zero and the current exponential ceiling. This spreads requests out when many workers hit the limit together. When Retry-After is present, preserve its minimum waiting period; any added jitter should extend that wait.

Give each request a clear retry budget

4. Limit both retry attempts and total elapsed time. A request should stop when either budget runs out. An interactive chat may need a short deadline, while a background document job can often wait longer. Respect cancellation so a closed browser tab does not leave unnecessary retries running.

Check whether your SDK already retries 429 responses. Adding another retry loop around it can multiply attempts unexpectedly. Choose one place to control the policy, or deliberately account for both layers. Before retrying operations with side effects, check the API’s idempotency support and documented behavior to avoid duplicate work.

Reduce incoming pressure with a shared queue

5. Put a concurrency limit in front of API calls. Start conservatively, queue excess work, and adjust using observed responses. If several application instances share one account limit, coordinate them through a shared limiter; separate local counters cannot enforce an account-wide ceiling.

• Reduce simultaneous requests when 429 responses become frequent.
• Shorten prompts and requested output lengths when token usage is the limiting factor.
• Reuse suitable cached results and avoid sending duplicate jobs.
• Separate background work from interactive requests so a large batch does not consume all available capacity.

Track request volume, token usage, queue time, and retry counts. These measurements help identify which control needs adjustment without guessing.

Keep retry rules explicit when using Relay365

6. Relay365 fits applications that want pay-as-you-go access to multiple model families through a relay desk. It works with Claude Code, Codex, Gemini CLI, and OpenAI SDK clients. Its API base is https://api.api365relay.com; OpenAI clients use https://api.api365relay.com/v1.

If you use models such as Claude Haiku 4.5 and Gemini 2.5 Flash, check the applicable limits and error details for each route. A shared client configuration does not establish identical limits across models. Only switch models when the replacement meets the task’s quality, cost, and data-handling requirements.

Create a Relay365 account if that access pattern suits your project, then test your retry behavior with a small workload before increasing traffic.

Show users what happens after retries stop

7. When the retry budget is exhausted, explain that the service is temporarily busy and offer a later retry. For queued jobs, preserve progress and show their status. Handle quota or account errors separately with a clear corrective action. Your application should remain understandable even when a request cannot finish immediately.