Naplen

Why one code reviewer is not enough

By Kyle Pierce ·

What a single reviewer misses

My engineering runs on my own agent factory, which I call the pump. I do not rely on a single reviewer, because each reviewer only knows what it examined, and a plausible-sounding finding is easy to produce and easy to believe.

How a change is reviewed

  1. One digest of factsWhat changed and what it had to satisfy
  2. Two fresh-context reviewersAdversarial and simplifier passes, no edits
  3. Adjudicator reads the codeEvery finding checked against the diff
  4. Fix, then review the fixOnly a confirmed Blocking finding opens a round
  5. Fresh fixer, then meWhen the loop does not converge
Each step runs in its own session. The loop is bounded, and the last step is me.

Keep the author out of the review

Reviewers run with fresh context, on a harness that is not the author’s, and they make no edits. A reviewer that shares the author’s context inherits the author’s rationalizations. A reviewer that can edit will quietly patch what it should have escalated. So the reviewer reads the diff as its input and reports, and nothing else.

Two passes asking two questions

Each review has an adversarial pass and a simplifier pass. The adversarial pass looks at correctness and risk. The simplifier pass looks at proportionality: whether the change introduced more abstraction than the problem needed. It is a separate pass because over-engineering is not a bug and never fails a test.

Start from facts, not opinions

Both reviewers start from one digest of what changed, what the change had to satisfy, and what the previous round already judged. The digest records facts and carries no opinion. A digest that carries an opinion hands both reviewers the same opinion before either has looked.

Adjudicate against the code, not the reports

An adjudicator reconciles the two reports. It has to read the code and not only the reports: an adjudicator given only the review texts can detect that two reviewers disagree but not which one is right.

Agreement between two reviewers is not verification either. Two reviewers can share a blind spot, and a plausible-sounding finding is exactly the kind both produce. So every finding is checked against the diff, and the output is sorted into four buckets:

confirmed   verified against the diff
disputed    the reviewers disagree and the code does not settle it
rejected    checked against the diff and found not to hold
gaps        what no single reviewer could produce

A gap is what no single reviewer can produce, because each of them only knows what it examined.

Severity follows evidence

Every reviewer and tool reports into one severity scale:

Blocking
Worth reconsidering
Minor
Holds up

A finding is Blocking only if it was reproduced, and only a confirmed Blocking finding opens a fix round.

Fixes get reviewed too, and the loop is bounded

The fixer reproduces the failure before fixing it: it re-runs the command or measurement the finding cites and confirms the failure. After the fix passes, it perturbs the fix by removing or disabling it and re-running the test, and quotes the failure.

Every fix then gets its own review pass before it is pushed, because a fix is the highest-risk change in the loop. Once one fix round has landed, remaining non-blocking findings are recorded and the gate opens.

The loop is bounded. After five rounds on one change, the diagnosis goes to one fresh-context fixer, and then to a human if that fails.

Fix the class, not the instance

One failure pattern taught me a rule. On a single merge request, three defect patterns produced roughly nine findings across four review passes, because each pass fixed one instance of a pattern and left its siblings. That is an internal anecdote and not a benchmark, but it is why my agents are told that after fixing any pattern they search the whole repository for that pattern and fix or justify every other occurrence before calling the work done.

Back to Insights

One review of how your team ships, a concrete plan, and you decide whether to continue.

[email protected]