AIQ AIQ
World 5: Is AI Fair? · Lesson 5.2.2

Who's Responsible?

Everything a teacher needs to deliver this lesson — pick your grade's script below once you've read the background.

Learning Objectives

Like every AIQ lesson, this one is a short core loop — a hook, a set of tappable "scenes," and a quiz — built to run in one 5–15 minute sitting, with the same three scenes and the same underlying idea delivered at four different depths. By the end, a student should be able to:

Teacher Background

You do not need a legal or computer-science background to teach this lesson well. Every example in "Who's Responsible?" is really asking the same question in a different costume: when an AI system's decision hurts someone, whose fault is it? That question turns out to be much harder than it sounds, and the lesson is designed to let students sit with that difficulty rather than hand them a tidy answer.

Why it's hard: the accountability gap. When a human driver runs a red light, the chain of cause and responsibility is short and familiar — the driver made a choice, and courts have centuries of practice assigning blame for exactly that kind of choice. A self-driving car's mistake doesn't have that short chain. The car's behavior was shaped by a programmer's code, by a company's decision to train the system on a particular set of driving data, by another company's decision to deploy that system on public roads, by whoever maintained the vehicle's sensors, and sometimes by the passenger's own choices (did they override the system, ignore a warning?). When something goes wrong, the harm is real, but the cause is smeared across many decisions made by many people at many different times. Researchers call this the accountability gap — a term that describes the mismatch between how AI harm actually happens (diffusely, across a chain of decisions) and how our legal and moral intuitions about blame usually work (a single person did a single thing).

Key point: The AI system itself is never "responsible" in a legal or moral sense. It has no legal personhood anywhere in the world today — accountability always traces back to a person or an organization somewhere in the chain, even when that chain is long and the tracing is hard. (In 2017 the European Parliament briefly floated giving certain autonomous systems a limited "electronic personhood" status; the idea was heavily criticized by legal scholars and was never adopted. No jurisdiction currently treats an AI system as a legal person that can itself bear responsibility.)

The "black box" problem. Many modern AI systems — the kind used to screen loan applications or recognize faces in images — work by adjusting millions of internal numeric settings until the system's outputs match a large set of known, correct examples. Once trained, the system can make new predictions, but there usually isn't a simple, human-readable explanation of exactly why it made one specific call rather than another. That opacity is what "black box" refers to: not secrecy, but genuine difficulty explaining a decision even when the company involved wants to. A research field called Explainable AI (XAI) works on tools that approximate an explanation after the fact — for example, highlighting which factors in a loan application weighed most heavily on a denial — but these are approximations of the system's behavior, not a transcript of its reasoning.

What's actually being done about it. The lesson names several real responses, and it's worth being able to describe each one in a sentence or two for your class:

The EU AI Act is a real law (it entered into force in 2024) that sorts AI systems into risk tiers — a small number of "unacceptable risk" uses are banned outright (like government social-scoring systems), "high-risk" uses (like hiring tools, credit scoring, and medical devices) must meet documentation, testing, and human-oversight requirements before deployment, "limited risk" systems like chatbots must disclose that a user is talking to AI, and everything else faces no special obligation. It is the first broad, binding law of its kind anywhere in the world, though — like any new regulation — the details of how it's enforced are still being worked out, so treat it as "here is the kind of law being written," not a fixed set of facts to quiz students on precisely.

The "right to explanation" the lesson mentions traces back to European data-privacy law (the GDPR), which gives people a right not to be subject to a purely automated decision with a significant effect on them, and a right to "meaningful information" about the logic involved. Worth knowing if a sharp student pushes back: legal scholars have actually debated for years how strong and specific this right really is in practice — it is real and it is influential, but it is not as simple or absolute as "the AI must show its work" might suggest.

Key point for the classroom: Other countries have taken different approaches rather than one global standard — the United States has historically leaned on existing agencies and sector-specific rules (financial regulators for lending AI, the FDA for medical AI) plus non-binding executive guidance rather than one comprehensive law; China requires companies to register certain algorithms with regulators and label AI-generated content; and UNESCO's 2021 Recommendation on the Ethics of AI was adopted by all 193 member states as a shared set of ethical principles, but it is a recommendation, not enforceable law, so how (and whether) each country acts on it varies widely. This patchwork is itself part of the lesson: there is no single global rulebook for AI yet.

For Hacker and Architect classes, three more concrete governance tools are worth having ready: model cards and datasheets for datasets are now-common practices where a company publishes a short, standardized document describing what a model or dataset is, what it was trained on, its known limitations, and its intended uses — essentially a nutrition label for an AI system. Algorithmic impact assessments are a pre-deployment review process — used by some governments already — that requires an organization to document a system's likely harms and fairness implications before it goes live, not after something goes wrong. And ISO/IEC 42001, published in 2023, is a formal international management-system standard for how an organization should govern its AI development responsibly, similar in spirit to existing standards for information-security management.

One honest note for older classes: the Philippines does not yet have a dedicated AI law. Some of this ground is covered indirectly by the existing Data Privacy Act (RA 10173) and guidance from the National Privacy Commission on automated decision-making, but there is no Philippine equivalent of the EU AI Act as of this writing. If a student asks, "does this apply to us?" — the honest answer is that the legal picture here is still developing, which is itself a fair and interesting thing for older students to sit with rather than something to treat as a gap in your knowledge.

Materials & Prep

Nothing beyond the AIQ standard: one device per student (or per small group) with a modern browser and an internet connection to reach aiq.ph. No login is required, no printouts, and no lab equipment. The lesson is entirely self-contained inside the app for all four age bands.

If you only have one shared device or a projector, this lesson works well as a teacher-led, whole-class read-along — open it on the shared screen, read or paraphrase each scene aloud, and use the age-specific delivery script linked below rather than relying on students clicking through independently. This is a natural fit for Explorer and Builder mode especially, since both scripts are written around teacher narration already. For Hacker and Architect classes, individual device access is more valuable, since the discussion and quiz for those age bands work best after students have had a moment to read and think at their own pace.

No vocabulary needs pre-teaching for Explorer or Builder. For Hacker and Architect classes, it's worth having a one-sentence definition of "black box" ready before you start — "a system whose internal decision-making can't be fully explained, even by the people who built it" — since every other idea in the lesson builds on that one.

Common Misconceptions

"The AI is responsible — it made the decision."
An AI system has no legal personhood anywhere in the world today. It cannot be sued, fined, or held to account in the way a person or a company can. When people say "the AI decided," that's a useful shorthand, but responsibility always has to trace back to a human or an organization somewhere in the chain — the people who built it, trained it, sold it, deployed it, or used it. If a student's answer stops at "the AI did it," gently push one step further: "okay, but who's the person or company behind that AI?"
"Black box" means the AI is literally hidden inside a physical black box, or that it's being kept secret on purpose.
"Black box" describes a system whose internal reasoning can't be fully explained in human terms — not a container, and not necessarily secrecy. Even a company that wants to be completely transparent about its own AI system may genuinely not be able to produce a simple explanation for one specific decision, because that's how the underlying math works. This is a real technical limitation, and it's exactly why "Explainable AI" research exists as its own field trying to work around it.
"The EU AI Act banned AI" or "made AI illegal."
The EU AI Act bans only a small, specific list of "unacceptable risk" uses (like government social-scoring systems). The vast majority of AI use falls into lower-risk tiers that face documentation, transparency, or no special requirements at all — the law regulates AI by risk level, it doesn't prohibit AI as a technology.
"If a system is too complicated for anyone to fully explain, then nobody can really be blamed when it goes wrong."
This is exactly backwards from how accountability is meant to work, and it's worth naming directly if a student arrives at it — complexity is a reason for stronger upfront requirements (testing, documentation, human oversight, audits), not an excuse that erases responsibility afterward. Every governance tool in this lesson — impact assessments, model cards, the EU AI Act's requirements for high-risk systems — exists precisely because "it's too complex to trace" is not an acceptable place to stop.

Pick your grade's script

← All lesson plans ← AI & the Environment Your AI Rules →