AIQ
AIQ
Part of the When AI Gets Confused lesson guide. Teaching a different grade? 🌈 Explorer (5–7) · 🔧 Builder (8–10) · 💻 Hacker (11–14)
Open with a framing question aimed at whatever these students are already using: "How many of you have used ChatGPT, Gemini, or a similar tool for schoolwork this month? Has it ever given you an answer that turned out to be wrong, or that you had to double-check?" Most hands will go up for the first question, and a meaningful number for the second. Say: "That's not a bug in one specific product — it's a structural property of how these systems are built, and today we're going to look at the actual failure modes underneath it, because if any of you go into a technical field, you'll be expected to understand — and design around — exactly these limitations." Then read the hook:
"AI systems fail in predictable ways. Understanding these failure modes is essential.
Let's examine adversarial attacks, hallucinations, and edge cases."
Set the register for the session explicitly: "We're not covering 'AI makes mistakes sometimes' as a warning label. We're covering four specific, named failure modes — adversarial examples, hallucination, distribution shift, and calibration failure — each with a real mechanism behind it, because 'be careful with AI' is not an actionable engineering principle, and by the end of this you should be able to explain, precisely, what can go wrong and why."
If your class includes students weighing career paths, this is a natural moment to say so directly: "Every failure mode we cover today has a corresponding job description — adversarial robustness researchers, AI safety engineers, model evaluation specialists, and AI policy analysts are all, in large part, people paid to think carefully about exactly these problems. This lesson is a legitimate first look at what that work actually involves, not just a warning to be careful."
Treat the three in-app scenes as the concrete case studies underneath a more rigorous framework, and spend the bulk of class time on the mechanism, not the anecdote — Architect mode students have the background to handle the actual technical explanation rather than an analogy standing in for it.
Scene 1 — Vision Fails: Adversarial Perturbations 👁️. Present the stop-sign attack and the banana-to-toaster example as instances of adversarial perturbation: a modern image classifier learns a high-dimensional decision boundary separating classes, and because that boundary is learned from finite training data rather than derived from first principles, there typically exist directions in input space where a small, carefully chosen perturbation — often imperceptible to a human observer — crosses the boundary and flips the predicted class. Note precisely what "imperceptible" does and doesn't mean here: the perturbation is small in a mathematical sense (small in pixel-value distance) and often small in perceptual terms too, but the key point for this age band is that the two notions of "small" are not the same thing, and adversarial robustness research is partly about which notion of distance actually matters for a given application. The leopard-print sofa example is a separate, related phenomenon — evidence that some image classifiers rely more heavily on local texture cues than on global shape or context, a bias introduced by what the training data and architecture happen to reward, not an attack someone engineered. On mitigation: adversarial training — deliberately including adversarial examples in the training set so the model learns to classify them correctly — is a standard countermeasure, but it typically trades off some accuracy on ordinary, non-adversarial inputs and doesn't generalize to every possible attack, which is why robustness against a specific, known attack is not the same claim as general robustness.
Scene 2 — Language Fails: Hallucination and Ambiguity 💬. Frame hallucination precisely: large language models are trained to predict the statistically likely next token given prior context, optimizing for fluency and plausibility, not for a ground-truth check against reality. There is no separate fact-verification subsystem built into the base training objective, so a confident, fluent, false statement is not a malfunction relative to that objective — it's the objective being satisfied in a case where "plausible" and "true" diverge. Cover sarcasm and lexical ambiguity ("bank") as failures of context modeling: correctly resolving them typically requires signals — tone, shared background knowledge, discourse context — that may be thin or absent in the input the model actually receives. On mitigation, mention this honestly rather than optimistically: techniques such as retrieval-augmented generation (grounding a model's answer in retrieved source documents rather than relying purely on what it memorized during training) and reinforcement learning from human feedback (further training that rewards outputs humans rate as accurate and helpful) measurably reduce hallucination rates in practice, but neither eliminates the underlying issue, because the base training objective — predict plausible text — is still what the model is doing underneath any of these additions.
Scene 3 — Why AI Fails: A Framework 🔍. Use this scene to introduce two terms the base lesson doesn't name but that Architect-level students should have: distribution shift (the training data doesn't fully represent the distribution of inputs the system will actually encounter at deployment, so accuracy degrades in the gap between them — "Data Limits" and "Context Blind" are both instances of this) and calibration (whether a model's stated or implied confidence tracks its actual probability of being correct — a poorly calibrated model can be just as confident when wrong as when right, which is arguably the most dangerous property covered today, since it removes the one signal — hesitation — that would otherwise tell a user to double-check). Close by connecting Defenses (🛡️) to real practice: adversarial training, red-teaming, and evaluation on out-of-distribution test sets are standard (though incomplete) parts of how responsible AI teams try to catch these failures before deployment, not a guarantee that they've been eliminated.
Spend a final few minutes making the distribution-shift/calibration relationship explicit, since students often conflate the two: distribution shift is a statement about the input — it has drifted from what the model was trained on — while calibration is a statement about the model's output — whether its stated confidence is trustworthy. A model can be well-calibrated on its training distribution and badly miscalibrated once distribution shift occurs, because nothing about calibration on familiar data guarantees calibration on unfamiliar data. That gap is a large part of why real-world AI failures often surprise the teams that built the system: their evaluation metrics looked fine on the data they tested against.
Close with: "None of today's failure modes are secret or unsolved-in-principle — they're active, well-documented areas of AI safety and robustness research, with real teams and real budgets working on them. If you go on to build or deploy AI systems yourselves, this is the baseline literacy you're expected to have: not 'AI can be wrong,' but knowing which specific failure mode you're up against and what mitigation actually addresses it."
Extension — Red-team a hypothetical system: In small groups, assign each group a plausible AI product (a resume-screening tool, a medical symptom checker, a content-moderation system, a customer-service chatbot, an automated grading tool). Each group produces a short "red-team memo": (1) identify which of today's failure modes — adversarial vulnerability, hallucination, distribution shift, poor calibration, spurious correlation — is most likely to cause real harm for that specific product, with a concrete scenario; (2) propose one realistic mitigation, acknowledging its limits rather than presenting it as a complete fix; and (3) state who should be accountable if the failure occurs anyway, and why. Have groups present in three minutes each, and press with follow-up questions during Q&A — this activity works best when treated like an actual technical review, not a presentation to be politely applauded.
For a class with more time or a stronger technical bent, add a research component: have each group find and cite one real academic paper or credible technical report documenting the failure mode they were assigned (adversarial examples, hallucination benchmarks, distribution shift, or calibration studies all have a substantial published literature), and require their proposed mitigation to reference what that source actually found rather than a general impression of the topic. This pushes the exercise from plausible speculation toward something closer to how an actual AI safety or evaluation team would scope a real risk assessment.