Curriculum/Beginner/Lesson 2 of 5

Beginner · lesson 2 of 5 · 11 min

How AI learns from data

AI systems are not programmed with rules. They are trained on examples. Learn what training is, why the data matters so much, and what a model can never learn from it.

What you will be able to do

  • Explain training in plain language
  • Describe why training data shapes what a model can do
  • Recognize when a model is being used outside what it learned

Learn · part 1 of 3

Examples in, patterns out

Traditional software follows rules a person wrote. A machine learning system is built differently: people collect a large set of examples, and a training process adjusts millions or billions of internal numbers until the system’s outputs match those examples closely. Nobody types in the rule for what a cat looks like or how a sentence should end. The model arrives at its own internal pattern from the examples it was shown, and that pattern is all it has.

In plain language
Training data

The collection of examples a machine learning system learns from. What the data includes, leaves out, and gets wrong determines what the resulting model can and cannot do.

Example

In a well-known 2016 experiment, researchers trained a system to tell wolves from huskies using photos in which every wolf happened to be standing in snow. It scored well. When they examined what it was looking at, the answer was the snow, not the animal. The model had learned exactly what the data taught it.

Learn · part 2 of 3

The data decides what the model can see

Because the examples are the only teacher, everything about them matters: where they came from, who and what is represented, how they were labeled, and how old they are. A model trained mostly on one group of people, one language, or one kind of situation will work best for that group and worse for everyone else. A model trained on text up to a certain date knows nothing after it. Problems in the data do not average out. They become part of the model.

Learn · part 3 of 3

Good on the test is not the same as good in the world

Builders hold back some examples to test the model on data it has not seen. A high score means the model handles new examples that resemble its training data. It says much less about situations that look different. A system can also score well for the wrong reason, by picking up on a shortcut in the data that happens to work in testing and fails in real use. This is why where and how a model was tested matters as much as the score.

Practice · try it yourself

Ask where it learned

Pick one AI system that makes or informs a decision about people, such as a resume screener, a loan model, or a grading tool, and work out what its training data would have to include to be fair to you.

You are not expected to find the real answers. The skill is knowing which questions to ask.

Name the system and the decision it informs
List what examples it was probably trained on
Identify who or what might be missing from those examples
Ask how old the data is and whether the world has changed since
Write the one question you would ask the vendor or owner

0 of 5 practice steps checked. Saved on this device.

Knowledge check · 2 questions

Check your understanding

Answer both to mark the lesson complete. You can try again. This is practice, not a graded exam.

1. What does it mean to train a machine learning model?

2. A model scores 95 percent in testing but performs poorly after launch. What is the most likely reason?

Recap · take it with you

Three things to remember

  1. Models learn patterns from examples; nobody writes the rules.
  2. Whatever is missing or skewed in the data is missing or skewed in the model.
  3. A test score only describes situations that resemble the training data.
Cover of AI for Everyone: No One Is Left Behind

Go deeper

AI for Everyone: No One Is Left Behind

Every idea in this path is expanded with more examples in the book.

View on Amazon ↗

Sources

  1. Google, Machine Learning Crash Course
  2. Ribeiro, Singh and Guestrin, “Why Should I Trust You?” Explaining the Predictions of Any Classifier (2016)
  3. Long and Magerko, What is AI Literacy? Competencies and Design Considerations (Georgia Tech, 2020)
  4. NIST, Towards a Standard for Identifying and Managing Bias in AI, SP 1270 (2022)

Examples are composites written for teaching unless a source is cited. Figures come from the sources listed where given.