Week 2, Part 1: Attention Itself
Causal Masking and Autoregressive Generation
Modify the attention implementation to apply a causal mask, and explain why a language model predicting the next token must not attend to future positions.
Modify the attention implementation to apply a causal mask, and explain why a language model predicting the next token must not attend to future positions.