Handling 429 Errors Without Making Them Worse

HTTP 429 means the server considers the client to have sent too many requests within a given period. Treat it as a signal to adjust traffic, not permission to retry immediately. ([rfc-editor.org](https://www.rfc-editor.org/info/rfc6585/?utm_source=openai)) For applications using CodeRelay Hub, put that handling around the shared API client, then apply the following walkthrough to each request path.

1. Classify the response before retrying

Inspect the status, structured error body, response headers, and request identifier when available. Do not classify failures by matching a single human-readable message. OpenAI, for example, distinguishes rate-limit failures from exhausted quota even though both can return 429. Waiting alone does not resolve exhausted credits or a spending limit. ([developers.openai.com](https://developers.openai.com/api/docs/guides/error-codes?utm_source=openai))

Create separate outcomes for temporary throttling, quota intervention, and unknown failures. For unknown responses, allow only a bounded retry policy and retain enough sanitized metadata for investigation. Never log API keys or complete prompts by default.

2. Honor the server’s waiting instruction

Parse Retry-After when present. Its standard forms are a nonnegative integer number of seconds and an HTTP date, not just a numeric duration. ([rfc-editor.org](https://www.rfc-editor.org/rfc/rfc9110.html?utm_source=openai)) Convert either form into a delay, validate it, and treat a past date as zero remaining wait.

Use a valid server delay as the minimum waiting period. If it exceeds the request’s remaining deadline, stop or defer the job instead of shortening the delay and retrying early. Treat missing or malformed values as a reason to use your local backoff policy.

3. Add capped exponential backoff with jitter

Without a usable server hint, choose a random delay between zero and a ceiling that grows exponentially after each failure. Cap that ceiling. Jitter spreads retries instead of having clients return together after identical waits. ([docs.aws.amazon.com](https://docs.aws.amazon.com/sdkref/latest/guide/feature-retry-behavior.html?utm_source=openai))

As an illustrative configuration, start with a one-second ceiling, double it after each retry, and cap it at 30 seconds. These are starting values, not provider limits. When Retry-After exists, wait at least that long; optionally add a small randomized delay afterward. Use asynchronous timers rather than blocking worker threads.

4. Give retries one owner and a deadline

Check your SDK before adding another retry loop. The official OpenAI Python SDK already retries selected failures, including 429 responses, by default. Configure or disable those retries deliberately when your application owns recovery. ([github.com](https://github.com/openai/openai-python/blob/main/README.md?plain=1&utm_source=openai))

Set both an attempt limit and an overall elapsed-time budget. Count the initial request explicitly, and include queue time, network time, and waiting time in the deadline. Propagate cancellation through every attempt. Choose one layer to own retries rather than independently retrying in the SDK, service wrapper, and job runner.

5. Control admission, not just recovery

Place a bounded queue and a concurrency limiter before outbound calls. Keep separate controls for simultaneous requests and request pacing. For token-sensitive workloads, also track estimated token demand and reconcile it against reported usage.

Scope limiters to the documented constraint: account, organization, model, or another published boundary. Coordinate shared limits across workers rather than giving every process the entire allowance. Reduce dispatch when throttling appears, then restore it gradually. OpenAI notes that unsuccessful requests can still consume its per-minute allowance, making immediate repeated retries counterproductive. ([developers.openai.com](https://developers.openai.com/api/docs/guides/rate-limits?utm_source=openai))

6. Make replay and fallback explicit

Distinguish a rejected request from an interrupted stream. Do not automatically append a fresh generation to partially delivered output. Define whether to discard, restart, or present the partial result, and track tool execution separately to avoid repeating side effects.

Allow model fallback only through an approved policy. Validate output requirements, tool support, privacy constraints, and cost controls first. Do not assume switching models bypasses a shared quota, and never rotate credentials to evade limits.

7. Test recovery before production

Use a fake clock and injected responses to test numeric and date-based Retry-After values, missing headers, repeated throttling, cancellation during sleep, deadline exhaustion, and queue overflow. Verify that retries stop exactly when configured.

Record logical requests separately from individual attempts. Track throttling frequency, retry count, queue age, total latency, and final outcomes. Alert on persistent throttling or growing queues, not merely isolated recovered errors. End interactive failures with a clear message; defer background work only when its deadline permits.

Open a free CodeRelay Hub workspace.