Teaching "How Computers Read" to Architect mode (ages 15–18)

Hook & Warm-Up

Open at the level this age band can actually engage with technically:

"From bag-of-words to Transformer architectures, NLP has evolved from statistical frequency analysis to contextual language understanding. Let's examine modern language representation and generation."

Ask: "Who here has used a large language model — ChatGPT, Claude, Gemini, whatever's on your phone? Now: can any of you actually describe, mechanically, what happens between you hitting 'send' and text appearing on screen?" Most students will describe the experience, not the mechanism. Say: "That gap — between using something fluently and understanding how it works — is exactly what today closes. We're going past 'it's smart' into tokenization, embeddings, attention, and the actual architectural difference between models like BERT and GPT."

Frame the stakes plainly, since this age band is old enough for it to matter directly: "Some of you will use these systems for schoolwork, some of you may end up building on top of them professionally, and all of you will need to evaluate what they get right and wrong for the rest of your lives. None of that is possible with 'it's magic' as your mental model — it requires knowing, at least at a working level, what's actually computing the output."

Main Activity

Treat this as a technical walkthrough building toward the Transformer, using the lesson's three scenes as checkpoints rather than the whole content.

Tokenization. Before any model sees text, it must be tokenized. Introduce Byte-Pair Encoding (BPE) and WordPiece as the two dominant subword tokenization schemes: both work by starting from individual characters and iteratively merging the most frequent adjacent pairs into single tokens, building a vocabulary that balances coverage against size — common words end up as single tokens, rare or novel words get split into smaller familiar pieces. This is why a model can process a word it's never seen before: it falls back to subword fragments it has seen.

Embeddings and positional encoding. Each token maps to a learned vector — a word embedding — in a high-dimensional space where geometric proximity approximates semantic similarity. The king − man + woman ≈ queen result is the standard illustration: consistent vector-arithmetic relationships emerge from training on raw text, with no explicit supervision on gender or royalty. One subtlety worth adding for this age band: because a Transformer processes all tokens in parallel rather than sequentially, it has no inherent notion of word order — positional encoding is added to each token's embedding specifically to inject that ordering information back in.

Self-attention and multi-head attention. This is the architectural core. Self-attention lets each token compute a weighted combination of every other token's representation, with weights determined by learned relevance — effectively, "how much should interpreting this token depend on that token." Walk through the classic disambiguation example on the board: "The trophy didn't fit in the suitcase because it was too big." Ask students to resolve what "it" refers to, then ask what would need to happen computationally for a model to get that right — the answer is that the representation for "it" needs to draw information from "trophy" specifically, across several intervening words, which is exactly what a learned attention weight between those two tokens accomplishes. Multi-head attention runs several of these attention computations in parallel with different learned parameters, letting the model capture different types of relationships simultaneously (say, syntactic dependency in one head and coreference in another). Layer normalization stabilizes training by normalizing activations between these sublayers, letting the very deep stacks used in real models train reliably.

BERT vs. GPT. Present these as two applications of the same Transformer building block, diverging on one key design choice: BERT is trained bidirectionally — it can condition on tokens both before and after the one it's predicting (via masked-language-model pretraining, where random tokens are hidden and the model learns to reconstruct them using surrounding context), which produces representations well-suited to classification and understanding tasks. GPT is trained autoregressively — each token is predicted using only preceding context, never future tokens — which is exactly the constraint that makes it usable for open-ended generation, since at inference time there is no "future" to condition on yet; you can't condition on text that doesn't exist yet. Close with fine-tuning: a pre-trained model, already holding broad language patterns from massive unlabeled text, can be adapted to a specific task using a comparatively small labeled dataset, which is why building a useful NLP system today rarely means training a model from scratch. It's worth naming the practical implication directly: pretraining on huge unlabeled text corpora is extremely expensive and done by a small number of well-resourced labs, but fine-tuning a pre-trained model is comparatively cheap, which is exactly what's made specialized NLP applications accessible to far more developers than a decade ago.

Worth flagging as an aside for this age band: the Transformer paper that introduced this architecture was titled "Attention Is All You Need" — the name itself was a claim that self-attention alone, without the recurrence or convolution earlier architectures relied on, was sufficient to achieve state-of-the-art results. That framing is a useful shorthand for what made the architecture such a significant departure from what came before it.

Run the practice round fast, then use it as a springboard: for each AI example (spam filter, auto-translate, autocomplete, chatbot), ask which of BERT-style or GPT-style behavior it more resembles — spam filtering and translation-quality scoring look like classification (BERT-family), autocomplete and chatbots look like generation (GPT-family), even though real production systems often blend both.

Discussion

• "Positional encoding exists because attention itself has no sense of word order. What would break in a sentence's meaning if that information were removed entirely?" (Push toward a concrete example: "the dog bit the man" and "the man bit the dog" contain identical tokens with identical attention relationships available — only word order distinguishes their meaning.)
• "Why does GPT's autoregressive constraint make it well suited for generation, but a worse fit than BERT for tasks like sentiment classification?" (Classification benefits from seeing the entire input at once before deciding; generation, by definition, produces output incrementally with no full input to condition backward from.)
• "Fine-tuning adapts a pre-trained model with a small labeled dataset instead of training from scratch. What does that imply about who can realistically build specialized NLP applications today versus ten years ago?" (The barrier to entry has dropped enormously — a small team can fine-tune an existing foundation model rather than needing the compute and data to pretrain one.)
• "If embeddings are learned purely from patterns in human-written text, what happens to biases already present in that text? Is there a clean fix?" (There isn't a fully clean fix — mitigation techniques exist, like debiasing embeddings post-hoc or curating training data, but none eliminate the underlying issue, since the model is fundamentally learning statistical regularities in whatever text it's given.)
• "Multi-head attention lets different heads capture different relationship types. Why might a single attention mechanism not be enough?" (A single attention pattern would have to average across every kind of relationship — syntax, coreference, topic — at once; separate heads let the model represent several distinct relationship types in parallel without them interfering.)
• "Few-shot prompting lets a model perform a new task from just a few in-context examples, without any weight updates. What does that suggest about what 'learning' means for these models, compared to how a traditional ML classifier is trained?"

Quiz Walkthrough

Wrap-Up & Extension

Close with: "Every fluent sentence a language model produces is the output of tokenization, learned embeddings, layered self-attention, and either bidirectional or autoregressive training — nothing more mystical than that, and nothing less genuinely sophisticated either. Understanding the mechanism is what lets you evaluate these systems critically instead of either worshipping or dismissing them."

Extension activity — architecture comparison table: Have students research and build a short comparison table (BERT vs. GPT vs. a third model of their choice — T5, LLaMA, or a current model they use) across: training objective, typical use case, context handling (bidirectional vs. autoregressive), and one real product built on it. Have a few students present their table and defend why a company might pick one architecture family over another for a specific product (say, a resume-screening tool vs. a customer service chatbot).

For a group with extra time or a strong technical baseline, add a second layer to the exercise: ask each group to identify one thing that could realistically go wrong if their chosen architecture were deployed for their chosen product without safeguards — a BERT-based resume screener trained on biased historical hiring data, or a GPT-based chatbot that confidently states an incorrect return policy. Require the failure mode to be traceable to something covered today (embeddings encoding real-world bias, or autoregressive generation optimizing for plausibility rather than truth) rather than a generic "AI can be wrong" statement. This extends the 10–15 minute core lesson into a full period and connects the architecture concepts to real engineering trade-offs and responsibilities, which is the level this age band should be operating at.