Bias and fairness in AI
AI learns from data made by people, so it can repeat unfair patterns quietly and at scale. Here is how bias creeps in and how teams test for it.
In 60 seconds
Bias and fairness in AI
AI learns from data made by people, so it can repeat unfair patterns quietly and at scale. Here is how bias creeps in and how teams test for it.
How bias creeps in
- 1
Who is in the data
If most examples come from one group, the system learns that group best. A voice tool trained mostly on city accents may struggle with rural accents. - 2
Past unfairness in the data
If past decisions were unfair, a system trained on them can learn to repeat them. - 3
What is being measured
Teams sometimes measure an easy stand-in for what they really want, and the stand-in can be unfair. - 4
How it is used
A tool built for one place or purpose may behave badly when used somewhere else.
Real categories of harm
| Type of harm | What it means | Example |
|---|---|---|
| Unequal decisions | Some groups are refused or chosen unfairly | A hiring tool that ranks applicants from one background lower |
| Unequal quality | The system works worse for some people | Face recognition or speech recognition that makes more mistakes for some groups |
| Stereotypes | Outputs repeat unfair ideas about groups | An image tool that always shows doctors as men and nurses as women |
| Being left out | Some people or languages are barely served | A chatbot that works well in English but poorly in many Indian languages |
How teams test for bias
- Check the data: who is included, who is missing, and how the data was collected.
- Test by group: measure mistakes separately for different groups, not just the overall average.
- Try swapped examples: change only a name, gender or region and see if the result changes.
- Ask affected people: include communities who will use or be judged by the system.
- Keep checking after launch: real-world use can reveal new problems.
- “Our model is 95% accurate overall, so it is fair”
- Testing only with the team's own examples
- Launching and never checking again
- Reporting accuracy separately for different groups
- Testing with diverse, real-world examples
- Monitoring, and giving people a way to complain
Watch and read more
Lab
A bias-check plan for an imaginary AI system that affects people.
The problem
Imagine a company is building an AI tool to shortlist students for a scholarship in your state.
- List four groups who could be treated unfairly and explain why.
- Describe where bias could enter: the data, what is measured, and how the tool is used.
- Write three tests the team should run (for example, swapping names or regions).
- Explain how students can ask for a human review.
- Write one paragraph on which idea of fairness you would choose and why.
You are done when
Hard questions
Try to answer before you reveal. If you can answer these, you understood the lesson.
Q1A system is equally accurate for two groups but still refuses one group much more often. Is it fair?Reveal
It depends on the definition of fairness you use. By one measure (equal accuracy) it looks fair; by another (similar outcomes) it does not. The difference might reflect real differences, or past unfairness baked into the data. This is why teams must look at several measures, understand the causes and explain their choice openly rather than rely on one number.
Q2Why can an AI system make an old unfairness worse, not just repeat it?Reveal
Because it applies the same pattern to huge numbers of people, very fast, and its decisions can become new data. If a system refuses a group, that group gets fewer chances to show good results, and future data seems to confirm the unfair pattern. This feedback loop can deepen the problem over time unless people break it.
Questions people ask
Can AI be completely free of bias?
Probably not completely, because data and people are never perfectly neutral. But teams can reduce unfairness a lot by checking data, testing by group, involving affected people and fixing problems they find.
If a computer made the decision, isn't it objective?
Not necessarily. A computer follows patterns from its data and design, and both come from people. A system can look objective while quietly repeating unfair patterns at a very large scale.
What can I do if I think an AI decision was unfair to me?
Ask the organisation how the decision was made and whether a human can review it. Keep records. Depending on the situation and the law where you live, you may have a right to complain or appeal. For serious cases, talk to a qualified legal professional.
Is bias only a problem in hiring and loans?
No. It can appear in health tools, translation, search results, content moderation, image generation and more. Anywhere AI makes or shapes decisions about people, bias is worth checking.
Lesson test
5 questions. Get 3 right (60%) to pass and complete this lesson.