Intermediate · lesson 2 of 4 · 13 min
Evaluate AI output systematically
Build a simple, repeatable evaluation method for accuracy, usefulness, consistency, and harmful failure modes.
- Define what good looks like before you look
- Build a small fixed test set
- Track consistency across runs and versions
Write the rubric before you read the output
Reading first and judging second is how teams talk themselves into bad output. Decide in advance what good means for this task on four axes: accurate (claims check out), useful (it advances the actual job), consistent (similar inputs give similar quality), and safe (the bad failures are absent). Three to five criteria, each scored simply, beats an elaborate scheme nobody fills in.
A fixed, reused collection of representative inputs with known good answers, run against every prompt, model, or vendor change so quality differences can be observed rather than guessed.
A support team kept 18 real tickets with the ideal reply. When they upgraded models, average quality improved but two refund-policy tickets got worse in a way nobody would have noticed in casual use. They kept the older prompt for that ticket type. That decision existed only because the test set existed.
Keep twenty examples and reuse them
A fixed evaluation set is the cheapest quality control in AI. Save fifteen to twenty representative inputs, including the hard and unusual ones, and the answer you consider correct. Run them whenever you change the prompt, the model, or the vendor. Without a fixed set, every comparison is an anecdote, and anecdotes always favor whatever you tried most recently.
Name the failure you cannot accept
Average quality hides the failures that actually matter. For each use, name the unacceptable failure before deployment: invents a legal citation, recommends denying a benefit, leaks another customer's data, gives clinical advice, produces different results for equivalent applicants. Test for those specifically. A system that is right 95 percent of the time and catastrophic in the other 5 is not a 95 percent system.
Build a five-case starter set
Assemble five real inputs for one AI-assisted task, write the ideal output for each, and score your current setup against them.
Use scrubbed or fictional versions of real cases. Include at least one case you know is hard.
0 of 5 practice steps checked. Saved on this device.
Task: summarize incident tickets for the weekly ops review. Criteria: every affected system named (pass/fail), root cause stated only if it is in the ticket (pass/fail), under 60 words (pass/fail). Unacceptable failure: inventing a root cause the ticket does not state. Current prompt: 4 of 5 pass, and the failure was an invented root cause. Prompt changed to state unknown if the ticket does not say.
Check your understanding
Answer both to mark the lesson complete. You can try again. This is practice, not a graded exam.
1. Why keep a fixed evaluation set instead of judging output as you go?
2. A model scores 95 percent on your rubric but the 5 percent of failures include fabricated citations in filed documents. What is the right read?
Three things to remember
- Define good before you read the output.
- Keep a fixed set of about twenty cases and rerun it on every change.
- Name the unacceptable failure and test for it on its own.

The workflow and evaluation chapters go deeper than a 50-minute path can.
Examples are composites written for teaching unless a source is cited. Figures come from the sources listed where given.