When an AI's answers lean one way because the data it learned from was lopsided.
An AI doesn't set out to treat anyone unfairly. But if the data it learned from leans towards certain genders, skin colours or places, its answers lean that way too. That tilt in an AI's judgements is called bias.
A 2018 study called Gender Shades found that commercial programs guessing gender from face photos made the most mistakes on darker-skinned women. When AI is used to choose between people, for jobs or loans for example, that tilt can cause real harm. Ways to reduce it include collecting more balanced data and checking results separately for different groups.
Think of it like this
Imagine choosing the school lunch menu by surveying only the class that loves spicy food. The menu might suit them, but not everyone else. An AI that learns mostly from one group of people can get things wrong for everyone else in the same way.
For example
If you ask for a picture of a "doctor" and always get people of the same gender and age, that could be a sign of bias.
Common mix-ups
Mix-upWhen AI leans one way, it's because it means to discriminate.
ActuallyEven with no intention, an AI that learns from past records copies past imbalances. Teach it what a "good applicant" is from old lists of who got accepted, and it may again pick fewer of the people who were picked less before.
Mix-upIf overall accuracy is high, it's accurate for everyone.
ActuallyIf 95% of the test data comes from one group, a model could be wrong for everyone in the other 5% and still score 95% overall. When you see an accuracy figure, also ask who it was tested on.
Try it in class
For teachers at the start of a lesson — or for students working on their own.
- Question
- In AI pictures of "a doctor", "a chef" or "a scientist", who shows up often and who's missing?
- Activity
- Each group picks one job and gives an image AI the same request five times, then tallies apparent features like age and gender in a table (no photos of real people). Pool the class's tables, discuss whether any lean shows up, and write down what harm it could cause if used to choose people — for jobs, say. Read what the Colorado record (E0370) made mandatory, too.
Records about this
Source-checked moments from AI history.
Widely cited as the first comprehensive state AI law in the US, but its obligations were never enforced: SB 25B-004 (signed Aug 28, 2025) pushed the date to June 30, 2026, and SB 26-189 (signed May 14, 2026) repealed and