Curriculum/Intermediate/Lesson 2 of 4

Intermediate · lesson 2 of 4 · 13 min

Evaluate AI output systematically

Build a simple, repeatable evaluation method for accuracy, usefulness, consistency, and harmful failure modes.

What you will be able to do

  • Define what good looks like before you look
  • Build a small fixed test set
  • Track consistency across runs and versions

Learn · part 1 of 3

Write the rubric before you read the output

Reading first and judging second is how teams talk themselves into bad output. Decide in advance what good means for this task on four axes: accurate (claims check out), useful (it advances the actual job), consistent (similar inputs give similar quality), and safe (the bad failures are absent). Three to five criteria, each scored simply, beats an elaborate scheme nobody fills in.

In plain language
Evaluation set

A fixed, reused collection of representative inputs with known good answers, run against every prompt, model, or vendor change so quality differences can be observed rather than guessed.

Example

A support team kept 18 real tickets with the ideal reply. When they upgraded models, average quality improved but two refund-policy tickets got worse in a way nobody would have noticed in casual use. They kept the older prompt for that ticket type. That decision existed only because the test set existed.

Learn · part 2 of 3

Keep twenty examples and reuse them

A fixed evaluation set is the cheapest quality control in AI. Save fifteen to twenty representative inputs, including the hard and unusual ones, and the answer you consider correct. Run them whenever you change the prompt, the model, or the vendor. Without a fixed set, every comparison is an anecdote, and anecdotes always favor whatever you tried most recently.

Learn · part 3 of 3

Name the failure you cannot accept

Average quality hides the failures that actually matter. For each use, name the unacceptable failure before deployment: invents a legal citation, recommends denying a benefit, leaks another customer's data, gives clinical advice, produces different results for equivalent applicants. Test for those specifically. A system that is right 95 percent of the time and catastrophic in the other 5 is not a 95 percent system.

Practice · try it yourself

Build a five-case starter set

Assemble five real inputs for one AI-assisted task, write the ideal output for each, and score your current setup against them.

Use scrubbed or fictional versions of real cases. Include at least one case you know is hard.

Collect five representative inputs, one of them hard
Write or paste the ideal output for each
Define three scoring criteria and a pass bar
Run the current prompt and score it
Write down the one failure mode you will not accept

0 of 5 practice steps checked. Saved on this device.

Knowledge check · 2 questions

Check your understanding

Answer both to mark the lesson complete. You can try again. This is practice, not a graded exam.

1. Why keep a fixed evaluation set instead of judging output as you go?

2. A model scores 95 percent on your rubric but the 5 percent of failures include fabricated citations in filed documents. What is the right read?

Recap · take it with you

Three things to remember

  1. Define good before you read the output.
  2. Keep a fixed set of about twenty cases and rerun it on every change.
  3. Name the unacceptable failure and test for it on its own.
Cover of AI for Everyone: No One Is Left Behind

Go deeper

AI for Everyone: No One Is Left Behind

The workflow and evaluation chapters go deeper than a 50-minute path can.

View on Amazon ↗

Sources

  1. NIST, AI Risk Management Framework 1.0 (2023)
  2. Anthropic, Define success criteria and build evaluations
  3. OpenAI, Evaluation guide

Examples are composites written for teaching unless a source is cited. Figures come from the sources listed where given.