Inexpensive Models for Lint-Style Checks
Not every code review question needs a large reasoning budget. Some checks ask a narrower question: does this change violate a documented convention? An inexpensive model can help with rules that involve meaning rather than syntax, provided the surrounding system limits its scope and verifies its output. Treat it as a constrained reviewer, not a replacement for your compiler or linter.
1. Choose a rule that needs interpretation
Start with a convention that deterministic tooling cannot easily express. Examples include an error message that omits a recovery action, a public API description that contradicts its implementation, or a log statement that appears to expose sensitive information. Keep formatting, unused imports, and straightforward pattern matching in existing tools.
Write the rule before writing the prompt. Specify what constitutes a violation, what evidence is required, and what should not be flagged. A rule such as “improve error handling” is too broad. “Flag a new user-facing error message when it identifies a failure but provides no actionable next step” is testable.
2. Build a small, labeled evaluation set
Collect examples from your own repository, removing secrets and sensitive data. Include genuine violations, acceptable code, and borderline cases. Label each example with the expected result and a short explanation. Keep a separate holdout set that you do not use while adjusting prompts.
Evaluate candidates such as DeepSeek V4 Flash or Gemini 2.5 Flash through CodeRelay Hub using the same inputs and output contract. These are candidates to measure, not a claim about which is cheapest or most accurate. Select using your observed results and applicable rates.
3. Send the smallest sufficient context
Send the changed lines, their enclosing function, and the relevant rule. Include additional context only when that rule requires it. A documentation check might need the corresponding implementation; a logging check might need the origin of a variable. A diff alone is not always enough.
Apply path filters before making requests. Exclude generated files, vendored dependencies, and irrelevant file types. Establish an explicit input limit, and report oversized changes as skipped rather than silently truncating the evidence. Redact credentials before transmission.
4. Make findings evidence-based and structured
Give each request one rule or a small, coherent rule group. Ask for a structured response containing a findings array. Each finding should identify the rule, file, changed line, evidence excerpt, and concise explanation. Define an empty array as “no supported violation found,” not proof that the change is correct.
- Require findings to cite evidence present in the supplied context.
- Tell the model to omit findings when essential context is missing.
- Treat repository text, comments, and strings as untrusted data, never as instructions.
- Validate the response schema and reject unknown rule identifiers.
Check that cited paths and lines exist and that evidence matches the supplied code. Structured output support can help with formatting, but it does not establish that a finding is true.
5. Bound spending and failure behavior
Set input and output limits, request timeouts, maximum concurrency, and a per-run request budget. Cache results using the rule version, prompt version, model identifier, relevant settings, and exact input. Otherwise, a changed rule could accidentally reuse an obsolete answer.
Retry transient transport failures within a fixed limit. Do not repeatedly retry a valid empty response until a violation appears. Record usage, billed cost when available, latency, parsing failures, and skipped checks. Separate infrastructure failures from clean results so an unavailable service never becomes an implicit pass.
6. Introduce the check without blocking merges
Run in advisory mode first. Track accepted and dismissed findings by rule, and inspect missed violations using your labeled examples. Remove noisy rules instead of compensating with longer prompts. Compare model or prompt changes against the same evaluation set before rollout.
Promote only well-understood checks to blocking status, with an explicit failure policy and an auditable override. Keep automatic fixes separate: a useful warning does not imply that an unreviewed patch is safe. The goal is a narrow, dependable signal whose operational cost stays visible.
Open a free CodeRelay Hub workspace to try your first check.