Set a Token Budget for Every Pull Request

An AI-assisted pull request can trigger several kinds of work: generating tests, explaining unfamiliar code, reviewing a diff, and revising a patch. Without a shared limit, each task can look reasonable while their combined usage grows unnoticed. A per-pull-request token budget makes that work explicit and gives automation a predictable stopping point.

1. Define what the budget covers

Start with a clear accounting boundary. Include every AI request made by your pull-request automation, including retries, follow-up reviews, and tool-driven continuations. Decide separately whether developer-initiated sessions belong in the same budget; attributing an entire interactive session to one pull request may be misleading.

Identify the budget with the repository and pull request number. Record the commit SHA on each request, but do not reset the budget whenever someone pushes another commit. Otherwise, repeated updates become an accidental escape hatch.

2. Separate token limits from spending limits

A token budget is not a currency budget. Different models and token categories can have different rates. Track input and output usage separately, retaining cache-related or other billable categories when the response exposes them. Do not blindly add every usage field: some fields describe subsets of a total.

If financial control matters, maintain a second ledger using your applicable rates. With CodeRelay Hub providing pay-as-you-go access through one endpoint, this distinction remains important when a workflow switches models. A shared endpoint simplifies integration; it does not make every token economically equivalent.

3. Allocate the budget by task

Choose an initial allowance from your own observed usage rather than an arbitrary industry target. Instrument several representative pull requests first. Inspect routine changes, dependency updates, and larger refactors separately before setting defaults.

• Preparation: Gather the diff and the minimum supporting context.
• Primary work: Reserve the largest share for the requested review or implementation.
• Verification: Leave room to inspect test failures or check a revised patch.
• Recovery: Keep a limited reserve for transient failures and explicitly approved retries.

These allocations are policy choices, not performance claims. Make them configuration values so teams can adjust the split without rewriting the workflow.

4. Trim context before reserving tokens

Exclude generated files, vendored dependencies, and irrelevant build artifacts unless the task specifically concerns them. Send the changed functions and necessary interfaces before sending entire files. For large diffs, divide the review into bounded units and preserve a compact summary between calls.

Estimate input size with a compatible tokenizer when available. Otherwise, treat your estimate as approximate and leave headroom. Set an explicit output limit too: a short prompt can still request an unnecessarily long answer. Ask for actionable findings rather than repeated explanations of unchanged code.

5. Reserve usage before dispatch

Enforce the budget in a central request wrapper, not just in a final reporting job. Before dispatching a request, atomically reserve its estimated input plus its allowed output and a safety margin. Reject or defer the request if that reservation exceeds the remaining allowance.

After completion, reconcile the reservation against reported usage. This prevents parallel CI jobs from independently spending the same remaining balance. Store a unique request identifier, task, model, reservation, and final usage. Keep unresolved reservations conservative when a timeout leaves the outcome unknown; do not assume that a missing response means nothing was consumed.

6. Make exhaustion useful, not mysterious

When the budget runs low, reduce scope before reducing review quality indiscriminately. Stop optional summaries, focus on changed code, or defer lower-priority checks. A smaller model such as Claude Haiku 4.5 can be a candidate for bounded preparation tasks, but validate suitability on your own workload.

If the allowance is exhausted, post a concise status explaining what completed, what remains, and how to request an increase. Never label an incomplete automated review as a successful full review. Require an explicit, logged override rather than silently replenishing the balance.

7. Review the policy alongside the results

Publish usage by task and revision in the pull request summary. Look for repeated context, expensive retry loops, and checks that produce little actionable output. Adjust allowances using both consumption and usefulness; low usage alone does not prove the workflow is effective.

Open a free CodeRelay Hub workspace to get started.