Reviewing Coding Agent Diffs: What to Check First
A coding agent can produce a plausible implementation before a reviewer has reconstructed the problem. That speed changes the review bottleneck: the hard part is deciding whether the patch preserves the system’s contracts. Senior engineers start with consequences, not formatting. Use this numbered walkthrough to review agent-generated diffs in risk order.
1. Establish the intended change before reading code
Read the issue, acceptance criteria, and relevant interface contract first. Write down the smallest observable behavior the patch should change, plus the behavior it must preserve. Otherwise, a convincing implementation can quietly redefine the task.
For a retry fix, ask whether retries apply only to transient failures, whether the operation is safe to repeat, and whether the existing timeout budget still holds. These questions give the review a concrete target.
2. Scan the file list for unexpected scope
Inspect the changed-file summary before reading individual lines. Look for configuration changes, dependency updates, schema migrations, generated files, and deleted tests. A small feature that also rewrites authentication deserves a scope discussion before detailed review.
- Explain every changed file. Ask how it contributes to the requested behavior.
- Separate mechanical edits. Formatting and renaming can hide meaningful changes.
- Inspect dependency churn. Check why both manifests and lockfiles changed.
Do not assume unrelated edits are harmless because an agent described them as cleanup.
3. Trace trust boundaries and authorization
Follow external input from its entry point to its sensitive use. Prioritize authorization checks, tenant identifiers, database queries, file paths, outbound requests, and secret handling. Validation that confirms a value’s shape does not establish permission to use it.
For example, an endpoint may correctly parse a project identifier while failing to verify that the caller owns the project. Check that authorization happens before data is returned or side effects occur, including in new helper functions and background jobs.
4. Check contracts at the callers
Read callers and implementations outside the diff. Agent-generated patches can look locally consistent while violating an assumption elsewhere. Search for changed functions, response fields, exceptions, configuration keys, and database columns.
If a function previously returned an empty list and now returns null, inspect every consumer. If an error becomes a successful response with an empty payload, verify that clients will not interpret failure as valid absence. Check defaults and serialization behavior, not just type signatures.
5. Walk through failure and concurrency paths
Trace what happens when a dependency times out, a request is duplicated, or a worker stops halfway through. Examine transaction boundaries and the order of durable writes versus external side effects.
A payment handler that records completion after sending a receipt can send duplicate receipts on retry. A cache invalidation placed before a database commit can expose stale data again. Ask which operations can repeat safely, which need idempotency protection, and how partial progress is recovered.
6. Read tests as claims, then challenge them
Check whether each test demonstrates a requirement rather than merely matching the implementation. An agent can reproduce the same mistaken assumption in both production code and tests.
- Add a negative case. Try unauthorized access, malformed input, or a missing resource.
- Exercise the boundary. Test empty collections, duplicate requests, and timeout exhaustion where relevant.
- Inspect mocks. Ensure they preserve the dependency behavior the patch relies on.
Where practical, confirm that a regression test fails against the previous implementation and passes with the patch. Run focused tests first, then the relevant integration checks.
7. Check deployment and rollback safety
A correct final state does not guarantee a safe rollout. For migrations, check compatibility while old and new application versions overlap. For configuration changes, verify missing-value behavior and deployment ordering. Confirm that logs expose failures without recording credentials or sensitive payloads.
Ask what happens if the application is rolled back after new data has been written. Destructive schema changes need a deliberate recovery plan, not just a passing test suite.
8. Make the approval evidence explicit
Finish with a short record of the behavior checked, tests actually run, and remaining uncertainty. Treat the agent’s summary as navigation, not proof. When comparing patches generated through CodeRelay Hub, apply the same review bar regardless of which model produced them.
Open a free workspace to put this checklist into practice.