AIQ
AIQ
Part of the When AI Isn't Fair lesson guide. Teaching a different grade? 🌈 Explorer (5–7) · 🔧 Builder (8–10) · ⚡ Architect (15–18)
Deliver the real hook line as a claim to be examined, not just a fact to absorb:
"Algorithmic bias isn't a bug — it's a reflection of the data, design choices, and societal structures encoded in AI systems. Let's examine how bias propagates and how to mitigate it."
Put this framing to the class directly: "Notice the wording — 'not a bug.' A bug is a mistake in the code. This is saying bias is something else: a working system doing exactly what it was built to do, faithfully reproducing a pattern that was already there. Does that make it more or less concerning than a simple coding mistake, and why?" Take a few opinions — there's no single right answer, and disagreement here is a good sign the class is actually engaging with the framing rather than just accepting it.
Push the point once more before moving on: "A bug usually gets fixed once, and then it stays fixed. If bias is a reflection of the data and the world rather than a coding error, does 'fixing' it even work the same way — is it a one-time patch, or something closer to an ongoing habit a team has to keep up?" Hold onto this question; the lesson answers it directly by the end.
Set up the throughline for the lesson: bias enters an AI system at three different points — the data it's trained on, the way the algorithm is designed, and how it's actually deployed and used in the real world — and a fix at only one of those three points is rarely enough.
Write the three points on the board as a quick reference the class will return to for the rest of the lesson: Data (what examples the system learned from), Design (what the system was built to optimize for, and what variables it was given), and Deployment (how it's actually used once it's out in the world, including by people who might use it differently than intended). Tell the class: "Every example today, we're going to ask which of these three it traces back to — and sometimes the honest answer is more than one."
Move through the app's three scenes, but push each one a level deeper than the facts on the screen — connect it explicitly to the "three sources" framing from the hook.
Present the resume-screening case and the Gender Shades face-recognition study (99% accuracy for light-skinned men, roughly 65% for dark-skinned women) as your two anchor cases. Ask the class to categorize each: is this primarily a data problem, a design problem, or a deployment problem? (Both are primarily data problems — unrepresentative training sets — which is a useful pattern to notice: most of the widely publicized AI bias scandals trace back to data, not to malicious or even careless algorithm design.) Bring in the lending example and introduce digital redlining by name — the practice, named for the historical maps banks used to literally mark neighborhoods red to exclude from lending, of an algorithm reproducing that same geographic exclusion without any human deciding to.
Push the categorization exercise one step further: ask whether the resume-screening case is only a data problem, or whether design played a role too. (It's both — the design choice to train the model directly on ten years of past hiring outcomes, rather than on some independently defined notion of "qualified," baked the historical pattern into the objective the model was optimizing for. A different design choice — say, training against a panel-reviewed definition of qualification instead of raw hiring outcomes — might have caught some of the bias even with the same data.) This is the point where "it's the data's fault" starts to feel too simple, which is exactly the nuance this age band should be reaching for.
This is where to introduce feedback loops, one of the harder and more important ideas in this lesson. Walk through an example: imagine a predictive policing AI that sends more patrols to a neighborhood based on past arrest records. More patrols in that neighborhood means more arrests get recorded there — not necessarily more crime, just more opportunities to record it. Those new arrest records then go back into the training data, "confirming" that the neighborhood needs even more patrols next time. Ask: "Where does this loop actually start, and where does it end?" (It doesn't have a clean end — that's the point of a feedback loop, and why catching bias early matters more than catching it late.)
Cover label bias and proxy variables as the data-and-design half of the picture: a proxy variable (ZIP code, school, surname) is often not chosen maliciously by a designer — it's just a feature that happened to correlate strongly with the outcome the model is being trained to predict, and correlation with a protected characteristic can ride along invisibly unless someone specifically checks for it.
Make label bias concrete with a specific scenario: a company hires people to label thousands of resumes as "strong candidate" or "weak candidate" so a model can learn from those labels. If the people doing the labeling unconsciously rate identical resumes differently based on the name at the top — a well-documented pattern in human hiring, independent of any AI — the model doesn't just repeat that bias, it learns it as if it were the actual, correct definition of a "strong candidate." The model is only ever as fair as the labels it was trained to imitate.
Present bias audits and diverse teams as institutional responses, then introduce a genuine technical countermeasure: adversarial debiasing. Explain it as a contest between two models trained together: the main model tries to make its normal predictions (say, "should this resume advance"), while a second model — the "adversary" — tries to guess a protected attribute (like gender) purely from the main model's internal predictions or representations. The main model is penalized whenever the adversary succeeds, which pushes it to stop encoding gender-correlated information at all, even indirectly. It's a genuinely elegant idea: instead of hand-picking which variables to remove, you train the system to actively resist leaking the sensitive signal.
Close the activity by naming AI impact assessments: structured reviews, done before a system launches and repeated on a schedule afterward, that check a system's real-world effects across different groups — not a one-time check but an ongoing practice, because a system that was fair on the data it launched with can drift as the world (and the data flowing into it) changes.
Connect this back to the feedback-loop idea explicitly: "This is exactly why a one-time bias audit before launch isn't enough. If a system starts a feedback loop after it launches, a check done only on day one would never catch it — you'd need to keep checking as new data comes in, which is exactly what an ongoing impact assessment is for."
Close with: "Bias in AI isn't a one-time bug to patch — it's a property of a system that keeps interacting with the real world, which means keeping it fair takes the same kind of ongoing attention as keeping it accurate."
Extension activity — "Design the Audit" (20–25 minutes): Split the class into small teams and give each team a fictional AI system relevant to the Philippines — a college scholarship-screening AI, a job-matching app for OFW applicants, or a credit-scoring tool for a microfinance lender. Each team designs a one-page bias audit plan: what groups would they test accuracy across (e.g., region, school type, gender), what data they'd need to collect to do that testing, and one proxy variable they'd specifically watch for in their scenario. Teams present their audit plan and the rest of the class tries to find a gap in it — a group the audit missed, or a proxy variable the team didn't consider. This stretches the lesson into a genuine design exercise and previews the kind of thinking a real ML engineer or policy reviewer does before a system ships.
If time allows, add a second round: have each team swap their audit plan with another team and specifically look for a feedback-loop risk — a way the system's own decisions could quietly shape the data that trains its next version. This pushes teams to apply the harder feedback-loop concept from the activity to a new case, rather than just recalling the policing example they were given, which is a much better test of whether the idea actually transferred.