Build Reliable Spend Alerts from a Usage Log
A usage log can tell you what happened. A spend alert should tell you when to act. Connecting the two requires more than summing costs on a schedule: retries, delayed records, and duplicate notifications can undermine an otherwise useful system. For teams using CodeRelay Hub, keep the alerting logic independent of whichever model handles a request. Build around a consistent cost record instead.
1. Define the spending boundary
Choose what each budget covers before writing the aggregation query. A workspace budget answers a different question from a project budget or an individual API key budget. Start with one scope and one currency, then add dimensions when someone needs a separate decision.
Specify the reporting timezone and period explicitly. A calendar-month budget should use calendar boundaries in that timezone, not a rolling thirty-day window. Store the resolved period start and end with each evaluation so you can reproduce an alert later.
2. Establish a trustworthy cost record
Inspect your actual usage export or logging pipeline rather than assuming particular fields exist. Your normalized record should include a stable event identifier, request identifier, workspace or project identifier, event timestamp, ingestion timestamp, currency, cost, and settlement status. Keep model and route identifiers when available for investigation.
Separate estimates from settled charges. If final costs arrive later, either update the original record or append a clearly linked adjustment. Do not sum both versions as independent charges. Store money as fixed-precision decimals or integer minor units with an explicit scale; avoid binary floating-point arithmetic for budget comparisons.
3. Make ingestion repeatable
Assume your importer will retry. Enforce uniqueness on a source event identifier, namespaced by its source when necessary. A request identifier alone may be insufficient if one request produces multiple charge components.
For append-only adjustments, retain a reference to the original event and allow signed amounts. For mutable records, require a version or update timestamp so an older delivery cannot overwrite a newer settlement. Test replaying the same input twice: the resulting spend must remain unchanged.
4. Aggregate by event time
Calculate period spend using the timestamp that your accounting policy assigns to the charge, not simply when your importer received it. Filter by budget scope, currency, and the agreed settlement policy before summing.
Delayed records require reconciliation. Recompute affected periods when late events or corrections arrive, and define how long closed periods remain revisable. If your source provides no reliable delay guarantee, do not describe recent totals as final. Display the latest successful ingestion time alongside the amount so readers can distinguish low spend from stale data.
5. Turn thresholds into persistent state
Use explicit, editable budget settings. For example, a team might configure an illustrative monthly budget of $100 and an initial warning at $80. These are policy choices, not API prices.
Persist each threshold crossing with a unique key containing the budget identifier, period start, threshold, and policy version. This prevents every scheduled evaluation from sending the same warning. Decide whether a correction below the threshold rearms the alert; a simple default is one notification per threshold per period.
Write the crossing and a pending notification into the same database transaction. A delivery worker can then retry notifications without losing the recorded decision.
6. Send an actionable message
Include the budget scope, period, current spend, threshold, evaluation time, and data freshness. Add the largest contributing projects or models only if your records support that breakdown. Give the recipient a clear action, such as reviewing an unexpected workload or lowering an application-side concurrency limit.
Do not include API keys, request bodies, or sensitive prompts. Also distinguish an alert from enforcement: sending a message does not stop requests. Any automatic restriction needs a separate, explicit policy and an independently tested enforcement path.
7. Test failures before enabling delivery
Run evaluations without sending messages first. Exercise duplicate imports, late charges, negative adjustments, month boundaries, stale ingestion, and notification failures. Confirm that simultaneous evaluators create only one crossing record and that retries preserve delivery state.
Track importer freshness, evaluation failures, pending notification age, and exhausted delivery attempts. An alerting system that silently stops running is another source of false confidence. Enable delivery once the totals reconcile with your chosen source of truth and the messages identify a concrete next step.
Open a free workspace in CodeRelay Hub.