AI for young mindsBeginnerLesson 773 min read

When AI is confidently wrong

AI can sound very sure and still be wrong. Learn why, and how to catch its mistakes.

✦ Ask the tutor🧪 Lab · 15 min📝 Lesson test
Lesson in motion

In 60 seconds

When AI is confidently wrong

AI can sound very sure and still be wrong. Learn why, and how to catch its mistakes.

1/5
In simple words
Imagine a classmate who never says "I don't know". Ask them anything and they answer confidently, even when they are guessing. Chatbots can be a bit like that.
A chatbot makes sentences that sound right, based on patterns in lots of text. It doesn't always check if they are right. When it makes something up, people call it a hallucination.

Common mistakes

  • Made-up facts: a wrong date, a wrong capital city, a person who never existed
  • Made-up sources: a book, website or quote that doesn't exist
  • Maths slips: small calculation mistakes, especially in long sums
  • Old information: things that changed recently
Real example
Meera asked a chatbot for a book to use in her project. It gave a title and an author. When she searched in the library and online, the book didn't exist. The chatbot had made it up, and it sounded completely real.

How to catch mistakes

  1. 1

    Ask "How do you know?"

    Ask for sources. Then check that those sources really exist.
  2. 2

    Check your textbook

    Your textbook and class notes are made for your syllabus.
  3. 3

    Do the maths yourself

    Or check with a calculator.
  4. 4

    Ask your teacher

    Especially if the AI and your book disagree.
Trusting too much
  • "It sounds sure, so it's right"
  • Copying a source without checking it exists
  • Believing it over your textbook
Smart checking
  • "Sounding sure is not proof"
  • Searching for the source yourself
  • Checking with your textbook or teacher
Note
AI isn't useless because it makes mistakes. People make mistakes too! It just means you check the important things.
Try this. With an adult, ask a chatbot a question you already know the answer to, like facts about your own town or school. Does it get everything right?

Watch and read more

Lab

A "catch the mistake" record comparing AI with your textbook.

~15 min

The problem

Do this with a parent or teacher (many chatbots are only for ages 13 and up).

  1. Pick 5 facts from your textbook (dates, definitions, a sum).
  2. Ask a chatbot the same 5 questions.
  3. Make a table: Question, Textbook answer, AI answer, Same or different?
  4. For any difference, check with your teacher or another trusted source. Who was right?

No chatbot? Do it on paper: an adult writes 5 "AI answers", some with sneaky mistakes, and you catch them.

You are done when

Hard questions

Try to answer before you reveal. If you can answer these, you understood the lesson.

Q1Why can a made-up source be more dangerous than a made-up fact?Reveal

A source makes an answer look checked and trustworthy. People may copy it into projects without looking. If the source doesn't exist, the mistake spreads further and looks more official than it is.

Q2Why might AI be wrong about something that happened last week?Reveal

Chatbots learn from text collected up to a certain time. Unless the app searches the internet, it may not know about recent events, and it might guess instead. For fresh news, check a trusted news source.

Please sign in to continue.

Questions people ask

Why doesn't the AI just say "I don't know"?

Chatbots are built to give helpful-sounding answers, so they often fill gaps with a guess. Some newer ones say when they're unsure more often, but you still can't count on it.

Is AI more likely to be wrong about some things?

Often yes: very recent news, small local facts, exact numbers, long calculations and quotes. Be extra careful there.

If two chatbots say the same thing, is it true?

Not necessarily. They may have learned the same wrong idea. Check with a trusted source like your textbook, a teacher or an official website.

Lesson test

5 questions. Get 3 right (60%) to pass and complete this lesson.

Sign in to take the test and save your progress