Ask a class whether an AI system learns from every person who uses it, and you will hear confident answers in both directions. Some students will say yes, of course. Others will say it just looks things up. Few will hesitate.
Confidence is the problem. A wrong idea that sounds fluent passes most tests. That is why formative assessment matters so much in AI lessons. The ideas are invisible, and students fill the gaps with guesses.
What one recent study found
Marx, Witt and Leonhardt (2024) interviewed five secondary school students about machine learning systems they meet every day: facial recognition and ChatGPT. The authors identified six misconceptions: Programmed Behavior, Exactness, Data Storage, Continuous Learning, User-trained Model, and Autonomous Data Acquisition. Two are easy to picture. Some students believed AI keeps learning while it is being used, or that training data is saved and reused later (WiPSCE ’24).
Ten-second checks that surface the idea
A good check takes about ten seconds and asks students to explain, not to pick a label. Here are five, starting with the two misconceptions from the study.
The first is the belief that AI keeps learning every time it is used. Ask:
A classifier is live and 100 people use it today. Has the model changed? What would have to happen for it to change?
A good answer separates training from use.
The second is the belief that training data is stored in the model and looked up later. Ask: “After training, where are the training photos? What does the model keep?” At school level, a good answer says the model keeps adjusted numbers (weights) shaped by the examples, not a folder of the examples themselves. Both misconceptions come from Marx, Witt and Leonhardt. The checks are our own suggestions, not the authors’. The second one links naturally to teaching why the examples matter in machine learning.
The next three come from APEX materials. In Grade 5, students meet the idea that high accuracy does not mean a model works everywhere. Ask: “The model scores 95% on its training photos and much lower on new ones. Which number do you trust?” A good answer points to test data and overfitting. In Grade 6, the Teacher Guide lists the misconception that a forward pass runs the model backwards. Ask students to draw the arrows and say which way the number travels. The answer is input to output. This is a good place to use explaining neural networks through small models. In Grade 8, the misconception is that the model understands what it says. Ask: “Why can a wrong answer sound so sure?” A good answer says the model predicts likely text and does not check facts.
Why small checks work better than big tests
- They are cheap. Two minutes, repeated often.
- They are timely. You find the idea before other ideas get built on top of it.
- They ask for explanation. “Explain why” reveals more than “true or false”.
- Wrong answers are data. Tally them, then re-teach the most common one.
They matter most where teachers are not AI specialists. If the misconception and the redirect are written down, the teacher does not have to improvise. See what non-specialist teachers need to teach AI confidently.
If a student’s wrong answer sounds fluent, how would your current assessment ever find it?
Build the check into the lesson
- Before the unit, list three to five likely misconceptions.
- End each session with one check tied to the idea just taught.
- Note the patterns, and open the next session by revisiting the most common wrong idea.
These checks are the quick end of a wider picture, covered in AI literacy assessment.
FAQs
How often should we run these checks?
In APEX, there is one named check per session. My rule of thumb is at least once for every new idea.
Should we correct a wrong idea straight away?
This is our view, not the study’s. Let students test their idea against evidence first by predicting, running the model and comparing. Then give the redirect.
Are these misconceptions common?
We don’t know. The study had five students, so use it as a list of things to check, not as a measure of how widespread they are.