AIQ
AIQ
Part of the When AI Isn't Fair lesson guide. Teaching a different grade? 🌈 Explorer (5–7) · 🔧 Builder (8–10) · 💻 Hacker (11–14)
Open with the lesson's hook line as a thesis to defend or challenge:
"AI bias isn't just unfair — it can cause real harm at scale. Let's examine how bias enters AI systems and what can be done about it."
Frame the stakes precisely for this age group, since most have started thinking about careers: "Everything in this lesson — hiring, lending, face recognition — describes decisions that used to be made one at a time, by individual humans, with individual biases that affected one applicant or one loan at a time. AI takes whatever bias exists and applies it identically, instantly, to millions of decisions. The word for that is scale. A human loan officer with a bias might unfairly reject a few dozen applicants over a career. A biased lending model can unfairly reject the same kind of applicant a million times before anyone notices the pattern." Ask: "Does turning individual bias into a scaled, quiet pattern make it easier to fix, or harder? Why?" (Harder to notice, but arguably easier to fix once caught, since you're auditing one system instead of retraining thousands of individual humans — both are defensible positions and worth debating.)
Add one more framing note before moving into the case review, since this age group is close to being the ones who will actually build or regulate these systems: "Notice that nothing in this lesson requires the AI to be advanced or futuristic. Every example — a resume screener, a face scanner, a lending model — is a fairly ordinary statistical system doing exactly what it was trained to do. The ethical weight here isn't about exotic AI capabilities. It's about the very ordinary fact that a system trained on the past will, by default, keep replaying the past, at scale, unless someone deliberately intervenes."
Use the app's three scenes as anchor cases, then formalize each with the actual technical vocabulary the field uses — this age band's summary explicitly promises "competing definitions" of fairness, and the activity should deliver on that.
Briefly re-establish the three real cases (biased resume screening trained on historically male hiring records; the Gender Shades face-recognition study showing large accuracy gaps by skin tone and gender; algorithmic lending patterns correlated with historically redlined neighborhoods). Then introduce the formal fairness vocabulary researchers use to evaluate systems like these:
Have students work in pairs on a short, concrete exercise: given a simplified fictional lending dataset (you can sketch this on the board — two groups, different approval rates, different accuracy rates), ask them to check whether the system satisfies demographic parity, equalized odds, and calibration all at once. It won't. That's the point, and it leads directly into the formal result below.
A workable board sketch: Group A has a 70% approval rate and the model is right 90% of the time; Group B has a 40% approval rate and the model is right 90% of the time too. Ask: does this satisfy demographic parity? (No — the approval rates differ.) Does it satisfy equalized odds? (Possibly, if the accuracy is genuinely equal both ways.) Now ask what happens if Group B's true repayment rate is actually lower than Group A's for reasons entirely outside the model's control, such as differing access to stable income documentation — forcing equal approval rates (parity) would then require accepting a higher error rate for one group, which breaks equalized odds. That trade-off, made visible with actual numbers, is the whole impossibility result in miniature.
Explain, at a conceptual level (no need for the underlying proof): mathematically, when two groups have different underlying base rates for the outcome being predicted, it is provably impossible for a system to simultaneously achieve calibration and equalized odds, except in special edge cases. This is sometimes called the fairness impossibility theorem, and it was established independently by multiple researchers around 2016–2017. It means fairness disagreements in real deployed systems are often not failures of technical effort — they can be genuine, unavoidable trade-offs, and the choice of which fairness definition to prioritize is a policy and values decision, not a purely technical one.
This is a genuinely uncomfortable result for students who came into the lesson assuming "fairness" is something engineers just haven't gotten around to solving yet. Sit with that discomfort rather than rushing past it: point out that this is one of the few places in the entire AI curriculum where more computing power, more data, or a smarter model does not make the underlying tension go away. The tension is baked into the different definitions themselves, not into any particular system's shortcomings.
Introduce counterfactual fairness as a different, complementary way to think about an individual decision: would this specific person have received a different outcome if, all else being equal, they had belonged to a different protected group? This reframes fairness around individual cases rather than group-level statistics, and it connects directly to U.S. legal doctrine on disparate impact — the principle, established in Griggs v. Duke Power Co. (1971), that a facially neutral policy or practice can still be found discriminatory if it disproportionately harms a protected group, regardless of whether that harm was intended. Point out to students that this legal standard predates AI by decades — it was built for human hiring and housing practices — and is now being applied, sometimes uneasily, to algorithmic decisions.
Close by naming what current best practice actually looks like: sociotechnical approaches that pair technical tools (bias audits, adversarial debiasing, calibration checks, open-source toolkits such as IBM's AI Fairness 360 or Google's What-If Tool) with institutional accountability (impact assessments, published audit results, regulatory review) and participation from the communities actually affected by the system, rather than treating fairness as something engineers can solve alone at their desks.
Make the "alone at their desks" point land with a direct question: "If the engineers who built a scholarship-screening algorithm had never met a single student who'd actually be affected by it, what's the risk? What could a mixed panel that included students, teachers, and policy staff catch that a purely technical team might miss?" This is the bridge from technical fairness metrics to the governance and participation half of sociotechnical thinking, and it's worth letting the discussion run a little before moving to the formal questions below.
Close with: "The math in this lesson doesn't have a clean happy ending — you genuinely cannot satisfy every fairness definition at once. What you can do, whether as an engineer, a policymaker, or just an informed citizen, is know which trade-off is being made, and insist that it's made on purpose, in the open, instead of by accident and left undiscovered."
Extension activity — "Fairness Trade-Off Memo" (25–30 minutes, or homework): Assign each student (or pair) a real or realistic AI system relevant to their own future — a college admissions algorithm, a scholarship allocator, a hiring screen for their intended field, or a Philippine context like an SSS or microfinance credit-scoring model. Have them write a one-page memo, addressed to a fictional product lead, that: (1) names which two fairness definitions are in tension for their system and why; (2) recommends which one to prioritize and defends that choice with a specific reason tied to the stakes of the system; and (3) proposes one concrete audit step to catch bias before launch. This mirrors the actual deliverable a junior ML engineer, product manager, or policy analyst might produce, and gives students a portfolio-ready artifact connecting directly to the "Societal Impact / Ethics" theme of World 5.
Encourage students heading toward computer science, law, public policy, or business to explicitly note which lens they're writing from in their memo — a CS-track student might lean on the technical audit step, while a policy-track student might lean harder on the disparate-impact and regulatory angle. Reading two or three memos aloud from different lenses at the end of class is a strong way to show that this problem genuinely needs more than one kind of expertise in the room, which is the sociotechnical point the lesson has been building toward all along.