Who is accountable when AI goes wrong
When an AI system causes harm, “the computer did it” is not an answer. Accountability means knowing who decides, who checks and how problems are reported and fixed.
In 60 seconds
Who is accountable when AI goes wrong
When an AI system causes harm, “the computer did it” is not an answer. Accountability means knowing who decides, who checks and how problems are reported and fixed.
Who is involved?
| Party | Typical responsibility |
|---|---|
| Model developer | Build and test the model, document its limits, reduce known risks |
| Company deploying it | Choose the right tool, test it for its use, train staff, monitor results |
| The person using it | Follow rules, check outputs, not misuse it |
| Regulators | Set rules, investigate problems, enforce laws |
| Auditors and researchers | Independently test systems and report problems |
Humans in the loop
- A staff member approves hundreds of AI decisions per hour without reading them
- The reviewer cannot see why the AI decided
- Overriding the AI is discouraged
- Reviewers have enough time and the right information
- The system explains its key reasons where possible
- Staff are trained and supported when they disagree with the AI
Audits: checking before and after
- 1
Before launch
Test for accuracy, bias, safety and security in realistic conditions. - 2
Document
Write down what the system is for, its limits and who is responsible. - 3
Monitor
Watch real results, complaints and unusual patterns. - 4
Independent review
Let outside experts test high-impact systems. - 5
Improve or stop
Fix problems, and be willing to switch the system off.
Reporting and fixing problems
- People affected should have a clear way to complain and ask for human review.
- Organisations should keep records so problems can be investigated.
- Serious incidents should be reported to the right authority where the law requires it, and lessons shared.
- Staff should be able to raise concerns without fear.
Watch and read more
Lab
An accountability map and incident plan for one AI system.
The problem
Choose an AI system used by an organisation, such as a bank's loan tool, a school's exam-proctoring tool or a city's traffic camera system.
- Draw a simple accountability map (a table is fine): list each party and its responsibility.
- Describe how human oversight should work: who reviews what, and when.
- Write a five-step incident plan: how problems are reported, checked, fixed, communicated and learned from.
- List three questions an independent auditor should ask.
- Explain one possible accountability gap and how to close it.
You are done when
Hard questions
Try to answer before you reveal. If you can answer these, you understood the lesson.
Q1Why can adding a human reviewer sometimes make accountability worse?Reveal
Because the organisation can point to the human as the one responsible, even if that person had no real chance to catch the error, for example because they had seconds per case or could not see the AI's reasons. This shifts blame onto the least powerful person. Real accountability requires designing the whole system so the reviewer can actually do the job, and keeping the organisation responsible for that design.
Q2Why are independent audits valuable even when a company tests its own AI carefully?Reveal
Because people naturally have blind spots about their own work and may be under pressure to launch. Independent auditors bring fresh eyes, different methods and less conflict of interest. Their findings can also build public trust. Internal testing and independent audits work best together.
Questions people ask
Can an AI system itself be held legally responsible?
Generally no. Legal systems hold people and organisations responsible, such as developers, companies using the system, or users. Debates about this continue, but the practical focus is on the humans and organisations behind the system.
What is an AI audit?
It is a structured check of an AI system: how well it works, whether it is fair and safe, whether it follows rules and whether it is documented properly. Audits can be done internally or by independent experts.
Does a human in the loop always make AI safe?
No. If the human is rushed, untrained or unable to disagree, the review is only for show. Meaningful oversight needs time, information, training and authority.
What should I do if an AI decision harmed me?
Ask the organisation for an explanation and a human review, keep records, and use its complaint process. If that fails, approach the relevant regulator or consumer forum. For serious harm, talk to a qualified legal professional.
Lesson test
5 questions. Get 3 right (60%) to pass and complete this lesson.