← All plans

Attention, Demystified: From Dot Products to Explaining Transformers

Goal: Understand how transformer language models work well enough to explain attention to a colleague · 2 weeks, about 4 hours per week

Explaining attention convincingly requires three things: fluency with the handful of linear-algebra operations transformers are built from, a mechanical understanding of how query/key/value projections turn a sequence into context-aware representations, and a mental model of how attention blocks stack into a working language model. You already have the programming ability to read and run reference implementations, so the gap is mostly conceptual vocabulary plus refreshed matrix intuition. This two-week plan spends the first week rebuilding the math and tracing a single attention head by hand and in NumPy, and the second week scaling up to full transformer blocks, training/inference behavior, and rehearsing a clear verbal explanation.

  1. 1Week 1, Part 1: The Math You Actually Need

    Reconstruct the specific linear-algebra operations transformers use — dot products as similarity, matrix multiplication as batched projection, and softmax as a weighting function — without needing a full linear algebra course.

    1. Dot Products as Similarity Scoresstarted
      Compute the dot product of two vectors by hand and explain why a larger dot product between normalized vectors means the vectors point in more similar directions.
    2. Matrix Multiplication as a Batch of Projections
      Given a (sequence_length × d_model) matrix and a (d_model × d_k) weight matrix, state the output shape and explain what each row of the result represents.
    3. Softmax and Turning Scores into Weights
      Apply softmax to a vector of raw scores by hand, verify the outputs sum to 1, and explain how scaling scores up or down sharpens or flattens the resulting distribution.

  2. 2Week 1, Part 2: From Text to Vectors

    Trace how a string of text becomes the numeric matrix that attention operates on, and explain why positional information must be added explicitly.

    1. Tokenization and the Embedding Table
      Explain how a subword tokenizer splits text into token IDs and how an embedding lookup converts those IDs into a sequence of vectors; predict the shape of the resulting matrix for a given input.
    2. Why Position Must Be Encoded
      Demonstrate that attention is permutation-invariant without positional information, and describe at least one scheme (sinusoidal or learned) for injecting position.

  3. 3Week 2, Part 1: Attention Itself

    Explain scaled dot-product attention mechanically — every matrix, every shape, every reason — and implement a single head in NumPy.

    1. Queries, Keys, and Values
      Describe the role of Q, K, and V using a retrieval analogy, and state which learned weight matrix produces each from the input embeddings.
    2. The Attention Formula, Line by Line
      Write out softmax(QKᵀ/√d_k)V from memory and justify each component, including why the scores are divided by √d_k.
    3. Implementing One Attention Head in NumPy
      Write a working single-head attention function in ~20 lines of NumPy and verify the attention weight matrix rows sum to 1.
    4. Causal Masking and Autoregressive Generation
      Modify the attention implementation to apply a causal mask, and explain why a language model predicting the next token must not attend to future positions.

  4. 4Week 2, Part 2: The Full Model and the Explanation

    Assemble attention into a complete transformer, understand what training and inference actually do, and deliver a clear five-minute explanation to a colleague.

    1. Multi-Head Attention
      Explain why multiple heads with smaller d_k are used instead of one large head, and trace how head outputs are concatenated and projected back to d_model.
    2. Inside a Transformer Block
      Diagram a full block — attention, residual connection, layer norm, feed-forward network — and state the purpose of each residual and normalization step.
    3. Training and Generation in Practice
      Explain next-token prediction with cross-entropy loss, and describe how a trained model generates text one token at a time, including the role of temperature in sampling.
    4. Rehearsing the Explanation
      Deliver a five-minute whiteboard explanation of attention using one worked example sentence, and answer three likely follow-up questions (why not RNNs, what the heads learn, why it scales quadratically with sequence length).