Beginner · lesson 3 of 5 · 10 min
How generative AI works
Learn why chatbots and image tools can create genuinely useful material, and why they can confidently invent details.
- Explain prediction in plain terms
- Name the three most common failure modes
- Decide when a draft is good enough to use
Prediction, one piece at a time
A large language model is trained on an enormous amount of text and learns which pieces of language tend to follow which. When you send a prompt, it produces the next piece, then the next, each time conditioned on everything written so far. Nothing in that loop checks a fact against the world. The loop optimizes for plausible continuation.
Output that is fluent and well-formed but factually fabricated, produced because the system optimizes for plausible language rather than verified truth.
Ask a model for five sources on a niche topic and you may get five perfectly formatted citations, two of which do not exist. The formatting is right because formatting is a pattern. The existence of the paper is not a pattern.
Why it invents things
Because the goal is plausibility, a gap in the model's knowledge is filled with the most plausible-looking material rather than an admission of ignorance. This is usually called hallucination or confabulation. It shows up most often in specifics: names, dates, page numbers, case citations, statistics, product features, and quotations. These are exactly the details people copy without checking.
Where it is genuinely strong
Generative AI is strong at transformation and at volume: rewriting for a different audience, summarizing material you supply, drafting a first version you will edit, generating alternatives, explaining a concept several ways, and translating between formats. Those uses put the model to work on content you already have, where you can see whether it got it right.
Find the seam
Ask an AI tool a question inside your own area of expertise, then mark every specific claim you can personally verify as true, false, or unverifiable.
Your own expertise is the safest test surface, because you can grade the answer without trusting another AI to grade it.
0 of 4 practice steps checked. Saved on this device.
A risk manager asked for a summary of the NIST AI Risk Management Framework. The four functions and their general purpose were described accurately. The number of subcategories under each function was wrong in three places, and one subcategory was invented outright. The pattern holds: prose was strong, enumerated specifics were weak.
Check your understanding
Answer both to mark the lesson complete. You can try again. This is practice, not a graded exam.
1. Why can a generative model state something false with total confidence?
2. Which task is the safest use of a general-purpose chatbot?
Three things to remember
- Generation predicts plausible continuations; it does not look up truth.
- Fabrication concentrates in specifics: names, numbers, citations, quotations.
- Use it where you can see the source and grade the output.

Every idea in this path is expanded with more examples in the book.
Examples are composites written for teaching unless a source is cited. Figures come from the sources listed where given.