Teaching "Good Data vs Bad Data" to Architect mode (ages 15–18)

Hook & Warm-Up

Open with the lesson's framing line, treated as a thesis statement to interrogate rather than accept at face value:

"The data you train on determines what your AI learns — including its biases. Let's examine how data quality shapes AI fairness and accuracy."

Ask the class: "If you were job-hunting in data science or ML engineering right now, and an interviewer asked you 'why does data quality matter more than model architecture for fairness,' what would you say?" Let a couple of students attempt an answer, then sharpen it: architectural choices (which algorithm, how many layers, which hyperparameters) can only optimize how well a model fits the patterns present in its training data — they cannot introduce information that was never in that data, and they cannot know, on their own, which patterns in the data are the ones you actually want the model to learn versus patterns you'd rather it ignore (like a spurious correlation with a protected attribute). That's why data quality and documentation have become their own professional discipline within ML, not just a preprocessing chore.

Frame the session as building toward a real skill: by the end of this lesson, students should be able to reason about a dataset the way a working data scientist or ML engineer does — asking about representativeness, provenance, and documented limitations before ever touching a model.

Run one more calibration question before opening the app: "If a company removes a protected attribute like race or gender from its training data entirely, is the resulting model guaranteed to be unbiased with respect to that attribute?" Most students will initially guess yes — it's an intuitive but incorrect assumption. Hold the answer for now ("no, and we'll see exactly why in a few minutes") — this sets up proxy discrimination as a genuine "aha" rather than a fact just handed to them.

Main Activity

Walk through the lesson's three scenes, using them as a scaffold to introduce the more advanced vocabulary from the age-band summary: representativeness, label accuracy, feature relevance, freshness, and the tools the field uses to document all of it.

Scene 1 — Data Quality ✨. Reframe the four items as named dimensions of dataset quality used in real ML practice:

• ✨ Clean Data / 🗑️ Dirty Data — this is label accuracy: how correctly examples are annotated. Even a small rate of systematic (non-random) labeling error can bias a model, because the model will learn the error pattern as if it were signal.
• 📏 Enough Data — insufficient sample size, which interacts with feature relevance: even a large dataset trains a weak model if the features it contains don't actually carry the information needed for the task.
• 🌈 Diverse Data — this is representativeness: whether the training distribution matches the deployment distribution. A model trained only on sunny-weather images has a representativeness gap for any real-world use in rain or fog.
• Introduce the fourth summary dimension not covered by the app's four items directly: freshness — data collected at one point in time can become stale as the real world changes, which connects directly to concept drift, covered below.

Scene 2 — Bias in Data ⚖️. Use the three real cases to introduce two more precise fairness concepts beyond what the Hacker-band script covers:

• Proxy discrimination. A model can end up discriminating against a protected group even if the protected attribute (like race or gender) was never included as a feature, because other, seemingly neutral features correlate strongly with it — a classic example is zip code correlating with race due to historical housing patterns. This is directly relevant to 🏦 Loan Bias: a lending model that never sees "race" as a field can still discriminate by race if it uses geography, and geography carries the same historical pattern race would have.
• Intersectional fairness. Bias doesn't only show up along single attributes — it can compound at intersections. Research on commercial face-analysis systems (most notably the "Gender Shades" work out of MIT) found that error rates were substantially higher for people at the intersection of two protected attributes (darker-skinned women) than for either attribute analyzed alone — worse than darker-skinned men, and worse than lighter-skinned women. This directly extends 👤 Face Bias: measuring fairness only along one axis at a time (skin tone alone, or gender alone) can hide a real problem that only appears at the intersection.
• 💼 Hiring Bias remains the clearest case of pure historical bias for grounding the general concept: it was reported that Amazon's experimental resume-screening tool, trained on ten years of male-dominated hiring data, taught itself to penalize resumes associated with women, and the company discontinued it once this was found.

Scene 3 — Fixing Data Problems 🔧. Elevate each item to the professional practice it corresponds to, and introduce the two major real-world documentation standards this age band's summary names:

• 🔍 Audit Data + 🧪 Test for Bias — together, these describe a fairness audit: measuring model performance disaggregated across protected groups and their intersections, often using metrics like demographic parity or equalized odds (the "Test for Bias" step is exactly the mechanism referenced by fairness metrics in the Hacker-band script).
• ➕ Add Diversity — in practice this is data augmentation and, increasingly, synthetic data generation. Be precise about the tradeoff the summary names: augmentation and synthetic data can meaningfully reduce underrepresentation, but they introduce their own risks — synthetic examples can encode the same biases as whatever generated them, and can create a false sense that a gap has been "solved" when the underlying real-world population still isn't well understood.
• Documentation. Two influential proposals formalize responsible data practice: Datasheets for Datasets (proposed by researchers including Timnit Gebru) call for datasets to ship with standardized documentation — how the data was collected, what populations it does and doesn't cover, known limitations, and intended uses — modeled on the datasheets that accompany electronic components. Model Cards (proposed by researchers including Margaret Mitchell) do the equivalent for trained models: documenting intended use cases, evaluation results across subgroups, and known limitations, so downstream users don't have to guess whether a model is appropriate for their situation.
• 👥 Diverse Teams — a process-level mitigation that documentation alone doesn't replace: people with different lived experience are more likely to identify a missing population or a harmful proxy variable that a homogeneous team, however well-intentioned, might not think to check for.

Return explicitly to the calibration question from the hook: "So — does removing a protected attribute guarantee an unbiased model?" The answer is no, and proxy discrimination is exactly why: correlated features can carry the same information the protected attribute would have carried, so the model can end up statistically discriminating on race, gender, or another protected characteristic without ever seeing it as a labeled field. This is precisely why fairness audits test outcomes across groups rather than just checking which fields were included in training — the presence or absence of a field in the schema tells you very little about whether the resulting predictions are fair.

Close the activity by having students articulate, out loud, the difference between representativeness, label accuracy, feature relevance, and freshness — the four dimensions named in this age band's summary — using one of the three real cases as an example for each.

Discussion

• "If you were an ML engineer and you inherited a model trained by someone else, with no datasheet or model card, what would you want to know before deploying it, and how would you go about finding that out?"
• "Why might synthetic data be an attractive fix for underrepresentation, and what specifically could go wrong if a team over-relies on it?"
• "Explain, in your own words, why removing a protected attribute like race from a dataset does not guarantee a model can't discriminate by race."
• "Concept drift means a model's accuracy can degrade over time even if nothing about the model itself changes. What real-world deployed system can you think of where the world plausibly changed faster than the training data was updated?"
• "Who should bear responsibility when a documented, audited model is later found to have a fairness problem the audit didn't catch — the engineers, the company, the auditors, regulators, or some combination? What would change your answer?"
• "A model card documents known limitations at the time it's published. If a model is later found to drift (its accuracy degrades as the world changes), does the original model card become inaccurate, and whose job is it to update it?"

Quiz Walkthrough

Wrap-Up & Extension

Close with: "Everything in this lesson — representativeness, proxy discrimination, intersectional fairness, datasheets, model cards — is current, practiced vocabulary in the AI industry, not classroom-only theory. If you go on to study or work in data science, ML engineering, or AI policy, this is the exact lens you'll be expected to apply before a model ever ships."

Extension activity — draft a real datasheet: Have each student (or pair) pick a hypothetical but specific AI application — a campus attendance face-scanner, a scholarship-recommendation tool, a local hospital triage assistant — and draft a one-page datasheet for its imagined training data, covering: how the data would be collected, which populations it would and wouldn't represent well, at least one plausible proxy-discrimination risk, and one fairness test they'd run before deployment. This is a genuine, portfolio-worthy artifact — the kind of document a working ML team actually produces — and stretches the core 5–15 minute lesson into a full, substantive class period with a tangible output students can keep.