Give Coding Agents a Dependable Unit Test Playbook

A coding agent can generate a convincing test suite that still misses the bug you care about. The fix is not simply asking for more tests. Give the agent a bounded workflow: discover the contract, identify failure modes, isolate dependencies, and prove the assertions matter. Whether you use an agent directly or through CodeRelay Hub, these instructions belong in the repository alongside your engineering conventions.

1. Start with repository discovery

Before requesting tests, require the agent to inspect the existing test runner, configuration, nearby tests, and contributor instructions. Tell it not to introduce a new framework or dependency without approval. Existing conventions often encode important decisions about fixtures, asynchronous cleanup, and environment setup.

Instruction: “Identify the smallest relevant test command and the established test patterns. Summarize them before editing. Do not change production behavior to make a test pass.”

2. Define the behavior before the examples

Ask the agent to state the public contract in plain English. For a retry helper, that might include which failures are retryable, how many attempts are allowed, and what happens after exhaustion. Implementation details are evidence to inspect, not automatically the specification.

If comments, callers, and implementation disagree, require the agent to flag the conflict rather than silently choose one. A regression test should preserve intended behavior, not accidentally canonize a defect.

3. Build a small risk-based case list

Have the agent propose cases before writing them. Prefer cases that distinguish correct behavior from plausible mistakes. For a parser, an ordinary input alone will not detect incorrect handling of an empty string or a missing delimiter.

• Cover the ordinary successful path.
• Exercise boundaries where behavior changes.
• Check invalid input and documented failure behavior.
• Add a regression case for the reported defect, if one exists.

Do not demand every combination of inputs. Ask the agent to explain which distinct failure mode each selected case targets.

4. Control sources of nondeterminism

Require fixed inputs for clocks, randomness, identifiers, environment variables, and external responses. Prefer existing dependency-injection seams or established framework utilities. Avoid real network calls and arbitrary sleeps in unit tests.

Instruction: “Use a controlled clock for time-dependent behavior. Restore modified environment variables and timers after each test. Await asynchronous work and verify cleanup.”

A fixed random seed helps reproduce randomized cases, but it does not control scheduling or remote services. Keep those sources of variability separate.

5. Mock boundaries, not the behavior under test

Tell the agent to use real domain logic inside the unit and replace expensive or uncontrollable boundaries. A mocked payment gateway can return a decline; the test should still execute the real decision logic that handles it.

Assert interactions when they are contractual, such as sending exactly one notification. Avoid asserting private helper calls merely because the implementation currently uses them. Also test an observable result: a returned value, a documented exception, or a meaningful state change.

6. Make assertions discriminate

“Did not throw” is usually too weak for a function that returns structured data. Require assertions on the fields that establish correctness, including important omissions or unchanged state. For errors, check the documented type or stable error code rather than a brittle full message unless that message is part of the contract.

Ask the agent: “What plausible incorrect implementation would this assertion reject?” If it cannot answer, strengthen the assertion or remove the redundant test.

7. Execute and challenge the tests

Require the focused test command first, followed by the relevant broader suite when feasible. For a bug fix, the new regression test should fail against the original behavior and pass after the fix.

When practical, temporarily introduce a plausible mistake, such as reversing a boundary comparison, and confirm the test fails. Revert that change immediately. This is a targeted assertion check, not proof that the suite catches every defect.

8. Demand an honest handoff

The final agent report should list changed files, behaviors covered, exact commands executed, and their outcomes. Separate verified results from assumptions. If execution is blocked by missing dependencies or credentials, say so explicitly; generated tests are not verified tests.

Open a free CodeRelay Hub workspace to put this playbook into practice.