Curriculum/Open topic/Lesson 1 of 4

Open topic · lesson 1 of 4 · 10 min

What is inside an AI data center

Every AI answer runs on physical machines in a physical building. Learn what those buildings contain and why AI changed them.

Rows of server racks and cabling in a data center

What you will be able to do

  • Explain the difference between training and inference
  • Describe why AI facilities need more power and cooling
  • Read a PUE figure correctly

Learn · part 1 of 3

Where AI actually runs

A chatbot feels weightless, but every answer is computed on servers in a data center: rows of computers, network equipment, electrical gear, and cooling systems inside a secured building. Two kinds of work happen there. Training builds a model once, over weeks or months, on thousands of specialized chips. Inference is every time someone uses the model, and it happens billions of times a day. As more people use AI, inference becomes a larger share of the energy bill.

In plain language
Power usage effectiveness (PUE)

Total facility energy divided by the energy used by the computing equipment. A PUE of 1.5 means that for every unit of energy the computers use, the building draws half a unit more for cooling, power conversion, and everything else.

Example

Take a campus whose computers draw 100 megawatts. At a PUE of 1.5, the whole facility draws 150 megawatts from the grid. At a PUE of 1.1, it draws 110. The 40-megawatt difference is roughly the electricity of tens of thousands of homes, from the same computing work.

Learn · part 2 of 3

Why AI changed the building

AI runs on specialized chips, such as GPUs and other accelerators, that draw far more power than ordinary servers and pack that power into less space. More power in a smaller space means more heat. That is why new AI facilities are larger, need more electricity, and increasingly pipe liquid directly to the chips. The International Energy Agency estimates that a typical AI-focused data center uses as much electricity as 100,000 households, and that the largest now under construction will use about 20 times as much.

Learn · part 3 of 3

Efficiency is measured, and it matters

The standard efficiency measure is power usage effectiveness, or PUE: the total energy a facility draws divided by the energy its computers use. A perfect score is 1.0. The Uptime Institute's surveys have put the industry average at about 1.56 for several years, while the most efficient large operators report figures near 1.1. Chips and software are also getting more efficient. Google reported a 33-fold drop in energy per median text prompt between May 2024 and May 2025, a figure the company notes was not independently verified.

Practice · try it yourself

Trace one AI answer

Pick an AI tool you use and find out what its provider publicly reports about the energy behind it.

Use the provider's own sustainability or environmental report. Note what is missing as carefully as what is there.

Name the AI tool and the company that runs it
Find the company's latest sustainability or environmental report
Look for a PUE figure or an energy-per-use figure
Write down what is not disclosed
Write one question you would ask the company

0 of 5 practice steps checked. Saved on this device.

Knowledge check · 2 questions

Check your understanding

Answer both to mark the lesson complete. You can try again. This is practice, not a graded exam.

1. A data center reports a PUE of 1.5. What does that mean?

2. Which kind of AI work happens billions of times a day?

Recap · take it with you

Three things to remember

  1. AI runs on physical buildings full of power-hungry, heat-producing chips.
  2. Training happens once; inference happens every time someone uses AI.
  3. PUE tells you how much energy goes to overhead. Closer to 1.0 is better.
Cover of AI for Everyone: No One Is Left Behind

Go deeper

AI for Everyone: No One Is Left Behind

A plain-language guide to AI for every mind, every age, and every community.

View on Amazon ↗

Sources

  1. International Energy Agency, Energy and AI, executive summary (2025)
  2. Google Cloud, Measuring the environmental impact of AI inference (Aug 2025)
  3. Uptime Institute, Global Data Center Survey (2025)

Examples are composites written for teaching unless a source is cited. Figures come from the sources listed where given.