Skip to content
FlintFlintby Torch
← LevelsBuilder · B9

Reviewer & QA agents

Build-along · ~12 min at your own keyboard

An agent that just wrote something is soaked in its own reasoning. A reviewer with a clean context reads the output cold and often catches things the writer skipped. But it's the same kind of model: it can share the same blind spots and the same wrong facts. So give it what makes it useful — the original ask, the source files, and explicit checks — and test it: plant a couple of known mistakes and see if it finds them. For anything with money, legal, or a client on it, a person still checks. Two more traps. Models lean toward agreeing with you, because they were trained on what people liked to hear. Ask 'is this good?' or say you wrote it, and the review comes back kinder. So ask neutrally: 'review this against these checks,' and don't say whose work it is. And a review is a judgement, not a measurement: run the same reviewer twice and it can flag different things. Treat one clean review as one opinion.

The long version: Reviewer subagents ↗

Do this

0/4

Checkpoints

  • ▸You should now see a reviewer spec that reads the deliverable, the original ask, and the sources, and reports issues only.
  • ▸You should now see whether the reviewer found your planted mistakes.
  • ▸You should now see a list of findings you sorted into real and false alarms.
The three ways this goes wrong
  • The reviewer rewrites the deliverable instead of reporting. — Tighten the spec: report issues only, never edit.
  • The reviewer says everything looks good. — Check that it got the original ask, not only the output. Without the ask it cannot judge fit.
  • The reviewer runs in the same session that wrote the work. — Start a new session or subagent. The point is a clean context.

Where these ideas come from: Sycophancy, Non-determinism (AI Hero dictionary). Paraphrased here.

Share what happened

Paste the planted-mistake result (found or missed), the reviewer's findings on the real deliverable, and which ones were legit.

Strip API keys, passwords, and tokens before you paste. Swap them for [KEY].

🔒 Don't paste confidential, personal-health, or payment info.