Powers of Ten

One forward pass, at four magnifications. Each level's totals are built from the level below rather than restated at it, so the levels cannot disagree — and the two places where the sums stop being exact are the two places memory traffic and wasted arithmetic come from.

Model

GPU

Phase

Sequence / context length

Batch size

Weight precision

FlashAttention (fuse the attention ops)

Model

What this level is made of

Does it add up?

Start at Model and zoom in. Click any operation in the layer breakdown to descend into it, or use the breadcrumb to jump.

Switch Phase from prefill to decode and watch the arithmetic intensity collapse. Nothing about the model changed — only how many query rows share the weights that had to be fetched.
Raise the batch size in decode. Intensity climbs roughly linearly, because the same fetched weights now serve more rows.
Drop the weights to INT4. Bytes fall, intensity rises, and an operation can flip from memory-bound to compute-bound without its FLOPs changing at all.
At Tile level, compare the bytes moved with the bytes an ideal machine would move. That ratio is the reuse factor.

Why don't the bytes add up at the tile level? Because a tile cannot keep the whole matrix. Every left-hand strip is re-read by each tile in its row, and every right-hand strip by each tile in its column. The arithmetic is counted correctly at every level; only the traffic grows as you look closer, and the factor by which it grows is what tiling exists to minimise.

Why do the FLOPs sometimes not add up either? A tile grid that does not divide the output evenly computes rows and columns that get thrown away. Small or awkwardly shaped matrices pay this; large ones barely notice.

Why is the model total not the model's parameter count × 2? That shortcut counts the layer stack, which is what this page sums. The embedding table and the projection onto the vocabulary sit outside it, and are excluded here rather than silently folded in.

Which numbers are these? The per-operation FLOPs and byte formulas are the ones the Layer Detail and Model Overview pages use; the tiled-traffic and per-tile figures are the GEMM Tiling Explorer's. This page adds the aggregation between them, not a new cost model.

GEMM Tiling takes the bottom level apart properly — tile shapes, wave quantization, and SM occupancy. KV Cache follows the memory that decode spends its time reading. Batching is the lever that moves decode back toward compute-bound. Cost turns all of it into dollars.