Week 2, Part 1: Attention Itself
The Attention Formula, Line by Line
Write out softmax(QKᵀ/√d_k)V from memory and justify each component, including why the scores are divided by √d_k.
Write out softmax(QKᵀ/√d_k)V from memory and justify each component, including why the scores are divided by √d_k.