AIQ
AIQ
Part of the Train Your Own Model lesson guide. Teaching a different grade? 🌈 Explorer (5–7) · 🔧 Builder (8–10) · ⚡ Architect (15–18)
Open with the lesson's real hook line: "Let's walk through the complete ML pipeline — from data collection to model deployment. You'll experience every step a real ML engineer goes through."
Ask the class: "When a company like Google builds an image-recognition feature, do you think an engineer sits down and writes rules like 'if it has pointy ears, it's a cat'?" Let students argue it out — some will guess yes. Say: "No — and today you're going to find out why that approach basically doesn't work, by building a real (tiny) version of the same kind of system those engineers build, using the exact same underlying technique: transfer learning."
Write on the board: data → train → test → deploy. Say: "Today we're not just clicking through these steps — we're going to look at what can go wrong at each one, using real terms: overfitting, train/test split, precision vs. recall, and hyperparameters. All four show up in today's quiz, and all four are things real ML engineers deal with constantly."
Quick diagnostic before opening the app: "If I train a model on 20 photos and then test it using those same 20 photos, and it gets 100% right, does that prove it actually learned to recognize the category — or could it just be memorizing?" Let the class debate. Say: "That exact question is why we never test a model on its own training data — hold that thought, we'll come back to it."
Push the framing one step further: "Every technique we'll name today — feature extraction, nearest-neighbor comparison, train/test splits, precision and recall — exists because someone ran into a real failure first and had to invent a fix. None of this is abstract theory taught for its own sake; it's a running list of 'here's what went wrong, and here's what fixed it,' built up over decades of people actually shipping these systems."
Work in pairs on the AI Playground, but treat each of the lesson's three scenes as an opportunity to name the real mechanism, not just perform the step.
Note the privacy design decision before starting: "Everything you capture — the raw photo and the feature vector computed from it — stays in this browser tab's own memory. Nothing gets uploaded to a server, and closing the tab wipes it all. That's a deliberate architecture choice given who this app is built for, not an accident — this whole pipeline runs entirely client-side."
Before capturing anything, explain what's really happening: "The app doesn't store your raw photos and compare them pixel-by-pixel — pixels are a terrible way to compare images, since the exact same object shifted one pixel to the left looks completely different at that level. Instead, it runs each photo through MobileNet, a neural network that was already trained on millions of general images, and pulls out a list of numbers describing the patterns MobileNet notices — edges, textures, shapes. That list of numbers is called a feature vector, and it's what actually gets compared, not the picture itself."
Have pairs collect at least 15-20 labeled examples per category, deliberately varying angle, distance, and lighting. Say: "You're not just taking pictures — you're populating the set of feature vectors the model will compare new photos against. A narrow, repetitive set of examples gives a narrow, unreliable set of feature vectors to compare with."
Add a technical aside for the students who push for more detail: "MobileNet, specifically, was trained to recognize roughly a thousand general object categories from a huge public dataset of labeled photos. It never saw your two categories during that original training. What it did learn is a general-purpose way of describing what's visually in an image — and it turns out those general-purpose descriptions are useful for telling apart categories the network was never explicitly trained on, as long as the new categories are visually distinguishable in the first place. That reusability across unrelated tasks is the whole reason transfer learning works."
Say plainly: "The app trains almost instantly, and here's why: it isn't running the 'training rounds' (epochs) with a slowly climbing accuracy number that the lesson describes in general — that describes classic neural-network training, where a model's internal weights get nudged a tiny bit after seeing each batch of examples, repeated many times. What's actually running here is closer to k-nearest-neighbor classification: a new photo's feature vector gets compared against every stored example, and it's assigned to whichever category its closest matches belong to. Both are real, legitimate ML techniques — the app picked the faster one specifically so it can run instantly in a browser tab with no server and no waiting."
Introduce hyperparameters here, tied to today's quiz: "A hyperparameter is a setting you choose before training starts — how many nearest neighbors to compare against, how long to train, how the data gets split — as opposed to something the model learns on its own from the data. The app has already picked reasonable defaults for you, but production ML teams spend real time tuning these."
Make the distinction concrete with a question: "One real hyperparameter in a comparison method like this is exactly how many of the closest stored examples to average together before deciding a category — just the single closest match, or several closest examples averaged together? The app's developers already fixed this choice for you, but imagine you could tune it: which do you think would be more resistant to one weird, mislabeled example accidentally throwing off a result?" Guide toward: averaging several nearby examples tends to be more robust to a single bad example than trusting only the one single closest match — which is in fact the approach the app uses internally.
Return to the opening diagnostic question. Say: "This is exactly why we test with photos the model has never seen — a proper train/test split. If we tested on training photos, a model that just memorized those exact images could score 100% while being useless on anything new. That gap between training performance and real performance on new data is called overfitting." Have pairs swap devices with another pair and test with genuinely new photos, tracking real accuracy honestly, including misses.
Introduce the precision/recall trade-off with a concrete framing: "Imagine your model sorts 'safe to eat' vs. 'not safe to eat.' Which mistake is worse — saying something unsafe IS safe (a false positive on 'safe'), or saying something safe is NOT safe (a false negative on 'safe')?" Let the class argue it through — most will land on the first being far more dangerous. Say: "That's the real trade-off engineers weigh constantly: optimizing to avoid one kind of mistake often means accepting more of the other kind. There's rarely a version of a model that has zero of both."
Close with: "Today you didn't just click through an app — you named the real mechanism behind every step: feature extraction instead of raw pixels, nearest-neighbor comparison instead of full training, a proper train/test split to catch overfitting, and the precision/recall trade-off that every real classifier has to navigate. That vocabulary is genuinely what you'd hear in an entry-level ML course or internship."
Leave them with an honest caveat about scale: "Everything we did today runs instantly, in a browser tab, on a couple dozen photos. A production system recognizing, say, defective products on a factory line would likely need thousands of carefully reviewed examples, a real held-out test set measured in the hundreds, and ongoing monitoring after deployment to catch when real-world conditions drift from what it was trained on. The four ideas are the same; the amount of engineering rigor around them scales up enormously with the stakes."
Extension activity — Break It On Purpose: In pairs, have students deliberately try to cause overfitting: train a model using only 3-4 nearly-identical training photos per category (same angle, same lighting, same background), then test it against photos taken in a different spot with different lighting. Have pairs record the accuracy drop and write one sentence explaining, in their own words, why a model trained on narrow data failed to generalize. Then have them retrain with the same total number of photos but deliberately varied conditions, retest, and compare accuracy — a direct, hands-on demonstration that data diversity, not data quantity alone, is what prevents overfitting.
If time allows, add a precision/recall twist to the same experiment: have pairs pick which category in their two-class model would be worse to misclassify (for example, if sorting "safe to touch" vs. "hot surface," a false "safe" is far worse than a false "hot"). Ask them to intentionally over-collect examples of the more dangerous category — training with, say, 25 examples of "hot" and only 10 of "safe" — and observe how the model's behavior shifts. Have pairs explain, using today's vocabulary, why deliberately imbalancing the data this way is a real (if crude) way to bias a model toward the safer kind of mistake.