Fresh-data testing

Build-along · ~10 min at your own keyboard

A model asked to test its own work generates test cases that flatter its own assumptions. The fix: test with data it has never seen — a real file from last month, a case you know is gnarly. Fresh data finds the failures polite tests never will.

The long version: Fresh-data testing ↗

Do this

0/4

Checkpoints

▸You should now see 3 real inputs it has never seen.
▸You should now see its outputs next to what you know is true.
▸You should now see one real failure named and fixed.

The three ways this goes wrong

You reused inputs from earlier sessions. — Pick files from last month or older, ones this session never saw.
Claude grades its own outputs as fine. — Grade them yourself against the facts. Do not ask it to mark itself.
Nothing fails. — Use a messier input: missing fields, odd formats, a refund, a blank.

Share what happened

Paste what fresh inputs you used, what broke, and the category of failure.

Strip API keys, passwords, and tokens before you paste. Swap them for [KEY].

🔒 Don't paste confidential, personal-health, or payment info.