AIQ
AIQ
Part of the When AI Gets Confused lesson guide. Teaching a different grade? 🌈 Explorer (5–7) · 🔧 Builder (8–10) · 💻 Hacker (11–14)
Open with a framing question aimed at whatever these students are already using: "How many of you have used ChatGPT, Gemini, or a similar tool for schoolwork this month? Has it ever given you an answer that turned out to be wrong, or that you had to double-check?" Most hands will go up for the first question, and a meaningful number for the second. Say: "That's not a bug in one specific product — it's a structural property of how these systems are built, and today we're going to look at the actual failure modes underneath it, because if any of you go into a technical field, you'll be expected to understand — and design around — exactly these limitations." Then read the hook:
"AI systems fail in predictable ways. Understanding these failure modes is essential.
Let's examine adversarial attacks, hallucinations, and edge cases."
Set the register for the session explicitly: "We're not covering 'AI makes mistakes sometimes' as a warning label. We're covering four specific, named failure modes — adversarial examples, hallucination, distribution shift, and calibration failure — each with a real mechanism behind it, because 'be careful with AI' is not an actionable engineering principle, and by the end of this you should be able to explain, precisely, what can go wrong and why."
If your class includes students weighing career paths, this is a natural moment to say so directly: "Every failure mode we cover today has a corresponding job description — adversarial robustness researchers, AI safety engineers, model evaluation specialists, and AI policy analysts are all, in large part, people paid to think carefully about exactly these problems. This lesson is a legitimate first look at what that work actually involves, not just a warning to be careful."
None of this is abstract for this room. The face-verification step in GCash and Maya, the road-sign-reading systems increasingly built into newer vehicles' driver-assist packages here, and the customer-service chatbots Philippine banks and telcos now deploy are all instances of the exact model classes this lesson covers — deployed at scale, on infrastructure many of you already use for actual money and communication. If any of you end up working in this field in the Philippines specifically, these are plausible systems you could be building, auditing, or stress-testing on purpose as part of the job.
Treat the three in-app scenes as the concrete case studies underneath a more rigorous framework, and spend the bulk of class time on the mechanism, not the anecdote — Architect mode students have the background to handle the actual technical explanation rather than an analogy standing in for it.
Scene 1 — Vision Fails: Adversarial Perturbations 👁️. Present the stop-sign attack and the banana-to-toaster example as instances of adversarial perturbation: a modern image classifier learns a high-dimensional decision boundary separating classes, and because that boundary is learned from finite training data rather than derived from first principles, there typically exist directions in input space where a small, carefully chosen perturbation — often imperceptible to a human observer — crosses the boundary and flips the predicted class. Note precisely what "imperceptible" does and doesn't mean here: the perturbation is small in a mathematical sense (small in pixel-value distance) and often small in perceptual terms too, but the key point for this age band is that the two notions of "small" are not the same thing, and adversarial robustness research is partly about which notion of distance actually matters for a given application. The leopard-print sofa example is a separate, related phenomenon — evidence that some image classifiers rely more heavily on local texture cues than on global shape or context, a bias introduced by what the training data and architecture happen to reward, not an attack someone engineered. On mitigation: adversarial training — deliberately including adversarial examples in the training set so the model learns to classify them correctly — is a standard countermeasure, but it typically trades off some accuracy on ordinary, non-adversarial inputs and doesn't generalize to every possible attack, which is why robustness against a specific, known attack is not the same claim as general robustness.
Two local cases are worth working through at the same level of technical detail. First, presentation attack detection (PAD) — the formal term for what GCash, Maya, and most banking apps implement as "liveness detection" during selfie verification — is a direct, deployed countermeasure against the same vulnerability class as the adversarial-glasses example: a face-recognition model checking only a single static image is trivially spoofable with a printed photo or a second phone displaying one. PAD systems typically require motion (a blink, a head turn, a spoken number) or a depth signal (a second camera, an IR sensor) precisely because those are harder for an attacker to fake than a flat image — though state-of-the-art attacks, including high-quality video replay, 3D masks, and increasingly real-time deepfake video injection, remain an active arms race in this literature. Ask: "PAD raises the cost of an attack rather than making it impossible. Is that a meaningful security improvement, or just theater?" (Defensible answer: raising attacker cost is a legitimate, standard security posture — very few real-world defenses are literally unbreakable, and the practical goal is usually pricing out casual attackers, not eliminating the most resourced ones.) Second, consider a road-sign-recognition or ADAS system trained mainly on clean reference imagery, deployed against actual Philippine road signage: sun-bleached reflective coating, rust, an informal ad sticker pasted over one corner, a sign bent by a passing jeepney. None of that is an engineered adversarial perturbation in the formal sense — nobody optimized the rust pattern against the model's gradients — but it produces the same practical failure through a different mechanism: distribution shift (covered in Scene 3) rather than a deliberate attack. Have students articulate the distinction precisely, since it's commonly conflated: an adversarial example is a small, deliberately optimized perturbation exploiting a specific model's decision boundary; a weathered sign is naturally occurring, undirected input drift relative to the training distribution. Both can flip a classification; only one required an attacker with access to the model.
Scene 2 — Language Fails: Hallucination and Ambiguity 💬. Frame hallucination precisely: large language models are trained to predict the statistically likely next token given prior context, optimizing for fluency and plausibility, not for a ground-truth check against reality. There is no separate fact-verification subsystem built into the base training objective, so a confident, fluent, false statement is not a malfunction relative to that objective — it's the objective being satisfied in a case where "plausible" and "true" diverge. Cover sarcasm and lexical ambiguity ("bank") as failures of context modeling: correctly resolving them typically requires signals — tone, shared background knowledge, discourse context — that may be thin or absent in the input the model actually receives. On mitigation, mention this honestly rather than optimistically: techniques such as retrieval-augmented generation (grounding a model's answer in retrieved source documents rather than relying purely on what it memorized during training) and reinforcement learning from human feedback (further training that rewards outputs humans rate as accurate and helpful) measurably reduce hallucination rates in practice, but neither eliminates the underlying issue, because the base training objective — predict plausible text — is still what the model is doing underneath any of these additions.
Code-switching is a useful case study here, because it's a genuine, still-unsolved NLP research problem rather than one with a known fix. A model's training data for Filipino and English are each individually substantial, but naturalistic Taglish — switching languages mid-clause, not just alternating between monolingual sentences — is comparatively underrepresented in most training corpora, including the ones many production systems are built or fine-tuned on: customer-service chatbots for Philippine banks and telcos, or SMS filters trying to catch smishing (phishing over text, like a fake "your GCash account is locked" message). Ask: "If a model handles pure English and pure Filipino well individually, why would you expect degraded performance specifically on the code-switched combination, rather than some average of the two?" (Because code-switching isn't two monolingual skills averaged together — correctly reading intent, sentiment, or a spam signal from a Taglish sentence sometimes depends on syntactic and pragmatic patterns unique to code-switched speech, which training data that treats the two languages separately doesn't capture well.) This is a live, low-resource-NLP research problem with direct local consequences: a smishing filter that underperforms on Taglish-phrased scam texts, or a bank chatbot that mishandles a Taglish complaint, isn't a hypothetical harm — it's the kind of gap that shows up in systems Philippine companies deploy today.
Scene 3 — Why AI Fails: A Framework 🔍. Use this scene to introduce two terms the base lesson doesn't name but that Architect-level students should have: distribution shift (the training data doesn't fully represent the distribution of inputs the system will actually encounter at deployment, so accuracy degrades in the gap between them — "Data Limits" and "Context Blind" are both instances of this) and calibration (whether a model's stated or implied confidence tracks its actual probability of being correct — a poorly calibrated model can be just as confident when wrong as when right, which is arguably the most dangerous property covered today, since it removes the one signal — hesitation — that would otherwise tell a user to double-check). Close by connecting Defenses (🛡️) to real practice: adversarial training, red-teaming, and evaluation on out-of-distribution test sets are standard (though incomplete) parts of how responsible AI teams try to catch these failures before deployment, not a guarantee that they've been eliminated.
Spend a final few minutes making the distribution-shift/calibration relationship explicit, since students often conflate the two: distribution shift is a statement about the input — it has drifted from what the model was trained on — while calibration is a statement about the model's output — whether its stated confidence is trustworthy. A model can be well-calibrated on its training distribution and badly miscalibrated once distribution shift occurs, because nothing about calibration on familiar data guarantees calibration on unfamiliar data. That gap is a large part of why real-world AI failures often surprise the teams that built the system: their evaluation metrics looked fine on the data they tested against.
Close with: "None of today's failure modes are secret or unsolved-in-principle — they're active, well-documented areas of AI safety and robustness research, with real teams and real budgets working on them. If you go on to build or deploy AI systems yourselves, this is the baseline literacy you're expected to have: not 'AI can be wrong,' but knowing which specific failure mode you're up against and what mitigation actually addresses it."
Extension — Red-team a hypothetical system: In small groups, assign each group a plausible AI product (a resume-screening tool, a medical symptom checker, a content-moderation system, a customer-service chatbot, an automated grading tool, an e-wallet's face-verification step, or a Taglish-aware SMS spam filter). Each group produces a short "red-team memo": (1) identify which of today's failure modes — adversarial vulnerability, hallucination, distribution shift, poor calibration, spurious correlation — is most likely to cause real harm for that specific product, with a concrete scenario; (2) propose one realistic mitigation, acknowledging its limits rather than presenting it as a complete fix; and (3) state who should be accountable if the failure occurs anyway, and why. Have groups present in three minutes each, and press with follow-up questions during Q&A — this activity works best when treated like an actual technical review, not a presentation to be politely applauded.
For a class with more time or a stronger technical bent, add a research component: have each group find and cite one real academic paper or credible technical report documenting the failure mode they were assigned (adversarial examples, hallucination benchmarks, distribution shift, or calibration studies all have a substantial published literature), and require their proposed mitigation to reference what that source actually found rather than a general impression of the topic. This pushes the exercise from plausible speculation toward something closer to how an actual AI safety or evaluation team would scope a real risk assessment.
You can use, adapt and share this guide for non-commercial purposes under CC BY-NC-SA 4.0, with credit to AIQ. Copy this line to give credit: