Code executes, the LLM judges
Build-along · ~10 min at your own keyboard
Models can give different answers to the same input; your invoice math shouldn't. The split: scripts handle counting, sorting, fetching, formatting (same answer every time, cheap, usually much faster); the model handles judgment — tone, prioritization, what matters. One caution: a script gives the same answer every time, including the same WRONG answer if its rule is wrong. Repeatable is not the same as correct, so check the script against a few rows you work out by hand.
The long version: Scripts for facts, models for judgment ↗
Do this
0/4Checkpoints
- ▸You should now see two columns, judgment and mechanical, with every step in one of them.
- ▸You should now see at least one mechanical step moved into a script.
- ▸You should now see identical output from two runs of the same input.
The three ways this goes wrong
- Everything lands in the judgment column. — Ask of each step: would two careful people get the same answer? If yes, it is mechanical.
- The script and the old model run disagree. — Check the script against a few rows by hand. The script is the one you can fix.
- The two runs differ. — Find the step still done by the model. Move the counting, sorting or filtering part of it into code.
Share what happened
Paste your two-column split, the twice-run consistency result, and your hand check of 3 rows.
Strip API keys, passwords, and tokens before you paste. Swap them for [KEY].
🔒 Don't paste confidential, personal-health, or payment info.