An attention mask marks which earlier token positions each token can use. The grid shows the allowed token pairs. A sparse GPU operation can skip fully masked blocks. The KV cache stores keys and values from earlier tokens.
Pattern
Sequence
Pattern Settings
KV Cache
Animation
Query × Key Mask Grid
computedmaskedpartial blockcurrent query
Dynamic KV Cache
Cost Signals
Mask Rule
Click a preset to change the settings.
Compare causal and sliding-window causal: the allowed grid shrinks from a triangle to a band, and usable KV history plateaus.
Switch MHA to GQA or MQA: the mask is unchanged, but the number of KV rows drops.
Use document masking to see why packed training batches cannot attend across document boundaries.
Increase the block size. Compare the fraction of masked cells with the fraction of fully masked blocks.
Why can a sparse GPU operation skip masked blocks, while dense attention still calculates every cell before applying the mask?
When does sliding-window attention reduce compute, KV memory, or both?
Why does GQA reduce KV cache rows without changing which token positions are legal?
Attention shows the query, key, and value calculation before masking.