The roofline model and arithmetic intensity

Before this lesson: lesson 1 (SMs, HBM, tensor cores) and the matrix product (linear algebra 1). For the attention rows of section 3 you need attention with a KV cache (deep learning 9; the KV cache is also treated in the LLMs track, in preparation); skip those rows if you have not seen attention yet. Time: about 60 minutes, plus the notebook.

---

Why this matters #

Before you optimise anything, you need to know what limits it. A two-line calculation tells you, and so what helps:

• memory-bound → move fewer bytes: fuse kernels (lesson 3), keep data on chip (lesson 4), use smaller number formats (lesson 11), batch more tokens per weight read (lesson 12);
• compute-bound → use tensor cores and lower precision, or do fewer FLOPs.

The same calculation explains why an H100 decodes an LLM only about 1.6× faster than an A100 (not 3×), and why a small MLP never gets close to the advertised FLOP/s. Every later lesson of this track uses it.

1. Predict first #

2. The picture #

Two resources. For one kernel, think of the GPU as a calculator and a pipe. The calculator does at most $P$ FLOP/s. The pipe moves at most $\mathrm{BW}$ bytes per second between HBM and the chip (lesson 1 explains why nearly all data crosses it). We use the H100 SXM (2022): $P = 989$ TFLOP/s dense bf16 on the tensor cores, 67 TFLOP/s fp32 on the CUDA cores, and $\mathrm{BW} = 3.35$ TB/s.

First case: an elementwise op. ReLU on $N = 10^8$ bf16 values reads $2N$ bytes and writes $2N$ bytes: $4 \times 10^8$ bytes in total. It does about one FLOP per value.

• Data: $4 \times 10^8\ \text{B} \,/\, 3.35 \times 10^{12}\ \text{B/s} = 119$ µs.
• Arithmetic, even on the slower CUDA cores: $10^8 / 67 \times 10^{12} = 1.5$ µs.

The calculator waits almost all the time. The op is memory-bound.

Second case: a square matrix multiply. For $n \times n$ matrices in bf16, $C = AB$ needs $2n^3$ FLOPs (one multiply and one add for each of the $n$ terms of each of the $n^2$ outputs). It moves at least $3 \cdot 2n^2$ bytes (read $A$ and $B$ once, write $C$ once). For $n = 4096$:

• Arithmetic: $2 \cdot 4096^3 / 989 \times 10^{12} = 139$ µs.
• Data: $6 \cdot 4096^2 / 3.35 \times 10^{12} = 30$ µs.

Now the pipe waits. The matmul is compute-bound.

The ratio decides. ReLU does $10^8 / (4 \times 10^8) = 0.25$ FLOP per byte. The matmul does $2n^3 / 6n^2 = n/3 \approx 1365$ FLOP per byte, because each number it loads is used $n$ times. This ratio is the subject of the whole lesson.

Interactive figure: loading…

3. Definitions and main results #

Intensity, ridge point and the roofline bound #

For one kernel on one GPU:

• FLOPs: the floating-point operations the kernel performs. One multiply-add counts as 2.
• bytes: the bytes it moves between HBM and the chip (reads plus writes).
• $P$: the peak FLOP/s of the GPU for the format and units used (dense, from the datasheet).
• $\mathrm{BW}$: the HBM bandwidth in bytes/s.
• arithmetic intensity $I = \text{FLOPs} / \text{bytes}$, in FLOP/byte. It is a property of the kernel (and its data sizes), not of the GPU.
• ridge point $R = P / \mathrm{BW}$, in FLOP/byte. It is a property of the GPU.

The roofline bound.

Why the two forms agree. Divide FLOPs by both sides of the first inequality. $\text{FLOPs}/t$ is at most $P$ and at most $\text{FLOPs} \cdot \mathrm{BW}/\text{bytes} = I \cdot \mathrm{BW}$.

On log–log axes (achieved FLOP/s against $I$), this bound is a sloped line $I \cdot \mathrm{BW}$ (the memory roof, slope 1) that meets a flat line $P$ (the compute roof) at $I = R$:

• $I < R$: the kernel is memory-bound; it can reach at most $I \cdot \mathrm{BW}$, a fraction $I/R$ of the peak.
• $I \ge R$: the kernel is compute-bound; it can reach $P$.

It is a bound, not a prediction. It assumes peak rates and a perfect overlap of arithmetic and data movement. Real kernels are slower. With no overlap at all, the time would be the sum of the two terms, and a sum of two numbers is at most twice their maximum. So the bound is a good first estimate: within a factor of 2 for a kernel that runs at peak rates.

Ridge points #

GPU and units	$P$ (dense)	BW	$R = P/\mathrm{BW}$
H100 SXM, bf16 tensor cores	989 TFLOP/s	3.35 TB/s	≈ 295 FLOP/byte
H100 SXM, fp8 tensor cores	1,979 TFLOP/s	3.35 TB/s	≈ 591
H100 SXM, fp32 CUDA cores	67 TFLOP/s	3.35 TB/s	20
A100 SXM 80 GB, bf16 tensor cores	312 TFLOP/s	2.039 TB/s	≈ 153
A100 SXM 80 GB, fp32 CUDA cores	19.5 TFLOP/s	2.039 TB/s	≈ 9.6

Elementwise ops and norms do a few FLOPs per element and move 4–6 bytes per element (in bf16): their intensity is below 5 FLOP/byte. That is below even the CUDA-core ridge of an A100. So they are memory-bound on every GPU in the table. This is why most non-matmul ops are limited by HBM bandwidth, and why a memory-bound op speeds up by the bandwidth ratio when you change GPUs, not by the FLOP/s ratio.

The matmul intensity theorem #

Theorem (matmul intensity). Let $X$ be $B \times D$ (activations: $B$ tokens of width $D$) and $W$ be $D \times F$ (a weight). In bf16, with each matrix moved once, $Y = XW$ has intensity

So $I \approx B$ for $B \ll D, F$, and $I \to DF/(D+F)$ as $B \to \infty$. On a GPU with ridge point $R$, the matmul is compute-bound exactly when

Why this is true, in one sentence. Each weight value is loaded once and used once per token, so when the weights dominate the traffic, the work per loaded byte grows with the number of tokens.

Proof.

1. FLOPs. Each of the $BF$ outputs is a dot product of length $D$: $D$ multiplies and $D$ adds. Total: $2BDF$.
2. Bytes. Read $X$ ($BD$ values), read $W$ ($DF$ values), write $Y$ ($BF$ values), 2 bytes each: $2(BD + DF + BF)$. No algorithm moves less, because each value must cross at least once.
3. Divide. The factor 2 cancels:

Divide the numerator and the denominator by $DF$: $I = B/(1 + B/D + B/F)$. 4. Limits. If $B \ll D, F$, the denominator is close to 1, so $I \approx B$: the weight read dominates. As $B \to \infty$, $I \to 1/(1/D + 1/F) = DF/(D+F)$: now the activation traffic dominates. 5. Compute-bound condition. Multiply both sides by $P$ and divide by bytes:

6. Solve for $B$. Multiply $I \ge R$ by the positive denominator $1 + B/D + B/F$:

If $1 - R/D - R/F > 0$, that is $DF/(D+F) > R$, divide to get $B \ge B^*$. Otherwise the left side is never larger than $R$, and no $B$ is large enough. $\blacksquare$

Rule of thumb. For large $D$ and $F$, $B^* \approx R$: a bf16 matmul on an H100 needs about 300 tokens per weight read to be compute-bound (an A100 about 150).

In fp8 (all three matrices in fp8), the bytes halve, so $I \approx 2B$; but the ridge point also doubles (591). The threshold stays at about 300 tokens: for $D = F = 4096$ it is $B^* = 345.2$, next to 344.9 in bf16.

Interactive figure: loading…

What "$B$" means in a transformer #

The $B$ of the theorem counts the tokens that share one weight read. It is not the number of sequences.

• Training and prefill: all tokens of the micro-batch go through each weight matrix together. $B = b \cdot s$ ($b$ sequences of length $s$), usually thousands of tokens. The weight matmuls are compute-bound.
• Decode (generating one token per sequence per step): each sequence brings one new token, so $B = b$, the number of sequences in the batch. With a few sequences, $I \approx b$: memory-bound.

Classifying the ops of a transformer #

For attention, take one head with head dimension $d_h$ and $T$ tokens in bf16, and count only the two matmuls $QK^\top$ and $PV$ (full attention, no causal mask).

Op	FLOPs	Bytes	Intensity	H100 regime ($R \approx 295$)
elementwise ops, norms	a few per element	4–6 per element	below 5	memory
weight matmul, training or prefill ($B$ = thousands of tokens)	$2BDF$	$2DF + 2B(D+F)$	up to $DF/(D+F)$	compute (if $D, F$ are large)
weight matmul, decode ($B$ = sequences)	$2BDF$	$\approx 2DF$	$\approx B$	memory while $B \lesssim 300$
attention, prefill, fused	$4T^2 d_h$	$8 T d_h$ ($Q, K, V, O$ once)	$T/2$	compute for $T \gtrsim 600$
attention, decode, MHA	$4 T d_h$ per sequence and head	$4 T d_h$ (its own $K$ and $V$)	$\approx 1$	memory, at any batch
attention, decode, GQA ($g$ query heads per KV head)	$4 g T d_h$ per sequence and KV head	$4 T d_h$	$\approx g$	memory

Batching does not help decode attention. Each sequence has its own KV cache. A larger batch adds bytes in the same proportion as FLOPs, so the intensity stays at about 1. GQA (grouped-query attention) helps because one read of a K/V head serves $g$ query heads; the cache is also $g$ times smaller.

Interactive figure: loading…

Answers to section 1. (1) About 1.6×, the bandwidth ratio $3.35 / 2.039 = 1.64$. ReLU is memory-bound, so the extra FLOP/s do not help. (2) About 300. For $D = F = 4096$ the exact threshold is $B^* = 344.9$, so $B = 345$ (section 4).

The back-of-the-envelope method #

Every performance question in this track starts with the same four steps:

1. Count the FLOPs (a matmul: $2BDF$; an elementwise op: a few per element).
2. Count the bytes that must cross between HBM and the chip (every input read once, every output written once, times the bytes per element).
3. Divide: $t_\text{compute} = \text{FLOPs}/P$ and $t_\text{memory} = \text{bytes}/\mathrm{BW}$.
4. Take the larger one. It is the minimum time and tells you the regime. Only a change that lowers the larger term makes the kernel faster.

Interactive figure: loading…

Limits of the model

• Caches. The bound counts the bytes that cross HBM. If data is read twice and fits in the 50 MB L2 cache, the second read may not reach HBM. The real HBM traffic can be lower than a naive count (or higher, if a kernel re-reads data that does not fit).
• Achievable peaks. Datasheet peaks are not reached in practice. A large, well-tuned bf16 matmul usually reaches about 70–80% of the datasheet FLOP/s, and a simple copy kernel about 80–90% of the datasheet bandwidth. Use measured peaks when you need precision.
• Several phases. A kernel with a memory-bound phase and a compute-bound phase takes about the sum of the two phase bounds, not one max.
• Launch overhead. Each kernel launch costs a few microseconds. For tiny tensors this overhead, not FLOPs or bytes, sets the time (lesson 3).
• Hierarchical roofline. You can draw one roof per memory level (HBM, L2, shared memory), each with its own bandwidth. A tiled kernel can be HBM-light but shared-memory-bound. Nsight Compute draws these charts.

4. Worked example #

A matmul at growing batch. $X$ ($B \times 4096$) times $W$ ($4096 \times 4096$) in bf16 on an H100: $P = 989$ TFLOP/s, $\mathrm{BW} = 3.35$ TB/s, $R = 989/3.35 = 295.2$ FLOP/byte. For each $B$:

• FLOPs $= 2B \cdot 4096 \cdot 4096 = 33.55\,\text{M} \times B$;
• bytes $= 2(4096B + 4096^2 + 4096B) = 33.55\,\text{MB} + 16{,}384\,B$ bytes;
• $I$ = FLOPs/bytes; the two times; the regime; the last column is the achieved fraction of peak at the roofline time, $\min(1, I/R)$.

$B$	FLOPs	Bytes	$I$	$\text{FLOPs}/P$	$\text{bytes}/\mathrm{BW}$	Regime	% of peak
1	33.6 M	33.6 MB	1.0	0.03 µs	10.0 µs	memory	0.3%
16	537 M	33.8 MB	15.9	0.54 µs	10.1 µs	memory	5.4%
64	2.15 G	34.6 MB	62.1	2.2 µs	10.3 µs	memory	21%
256	8.59 G	37.7 MB	227.6	8.7 µs	11.3 µs	memory	77%
345	11.6 G	39.2 MB	295.3	11.70 µs	11.70 µs	compute (just)	100%
1024	34.4 G	50.3 MB	682.7	34.7 µs	15.0 µs	compute	100%
4096	137 G	100.7 MB	1365	139 µs	30.0 µs	compute	100%

What to read from it:

• Batching is almost free below the ridge. From $B = 1$ to $B = 64$ the work grows 64×, but the time grows only 3% (10.0 → 10.3 µs): the 33.6 MB weight read is paid once.
• The exact threshold. $B^* = 295.2 / (1 - 295.2 \cdot 2/4096) = 295.2 / 0.856 = 344.9$, so $B = 345$, not the rule-of-thumb 295. The activation traffic $2B(D + F)$ (5.7 MB at $B = 345$, next to 33.6 MB of weights) is not negligible.
• On an A100 ($R = 312/2.039 = 153.0$): $B^* = 153.0 / (1 - 2 \cdot 153.0/4096) = 165.4$, so $B = 166$.
• A whole decode step. A 7B-parameter model in bf16 has 14 GB of weights. At batch 1, a decode step reads all of them once: $14 \times 10^9 / 3.35 \times 10^{12} = 4.2$ ms. So one sequence gets at most about 240 tokens/s, whatever the FLOP/s (A100: 6.9 ms, about 146 tokens/s).

5. In code #

The first block recomputes the worked example and checks the theorem against a brute-force search. NumPy only.

import numpy as np

P, BW = 989e12, 3.35e12          # H100 SXM: dense bf16 FLOP/s, HBM bytes/s
R = P / BW                        # ridge point, FLOP/byte

def matmul_cost(B, D, F, e=2):
    """FLOPs and minimal HBM bytes of (B x D) @ (D x F) with e bytes per element."""
    return 2 * B * D * F, e * (B * D + D * F + B * F)

def roofline_time(flops, nbytes, P=P, BW=BW):
    return max(flops / P, nbytes / BW)

for B in [1, 64, 345, 4096]:
    fl, by = matmul_cost(B, 4096, 4096)
    print(B, round(fl / by, 1), f"{roofline_time(fl, by) * 1e6:.2f} us")
# 1 1.0 10.02 us
# 64 62.1 10.33 us
# 345 295.3 11.71 us
# 4096 1365.3 138.97 us

def b_star(D, F, R=R):
    return R * D * F / (D * F - R * (D + F)) if D * F / (D + F) > R else np.inf

brute = next(B for B in range(1, 10**5) if np.divide(*matmul_cost(B, 4096, 4096)) >= R)
print(round(b_star(4096, 4096), 1), brute)            # 344.9 345
print(round(b_star(4096, 4096, 312 / 2.039), 1))      # 165.4   (A100)
print(b_star(512, 512))                               # inf     (D = F = 512 never compute-bound)

Your laptop CPU has a roofline too. This block measures one point far below the ridge and one far above it:

import time
import numpy as np

def best_time(fn, repeats=5):
    fn()                                          # warm-up
    times = []
    for _ in range(repeats):
        t0 = time.perf_counter(); fn(); times.append(time.perf_counter() - t0)
    return min(times)

# An elementwise op: 1 FLOP and 8 bytes per fp32 element (read x, write y).
n = 2**25
x = np.random.default_rng(0).standard_normal(n, dtype=np.float32)
y = np.empty_like(x)
t = best_time(lambda: np.multiply(x, 2.0, out=y))
print(f"elementwise: {n / t / 1e9:7.1f} GFLOP/s, {8 * n / t / 1e9:6.1f} GB/s, intensity 0.125")

# A square matmul: 2m^3 FLOPs and 12m^2 bytes (fp32), intensity m/6.
m = 2048
a, b = np.ones((m, m), np.float32), np.ones((m, m), np.float32)
c = np.empty((m, m), np.float32)
t = best_time(lambda: np.matmul(a, b, out=c))
print(f"matmul:      {2 * m**3 / t / 1e9:7.1f} GFLOP/s, {12 * m**2 / t / 1e9:6.1f} GB/s, intensity {m / 6:.0f}")

On the Apple M1 laptop where this lesson was checked, it printed 5–7 GFLOP/s at 40–60 GB/s for the elementwise op (the memory bandwidth) and about 900 GFLOP/s for the matmul. Your numbers will differ. The gap of more than 100× will not.

6. Where it shows up #

• LLM decoding is memory-bound, so servers batch many sequences (lesson 12), shrink weights and caches (lesson 11) and use GQA. Speculative decoding verifies several draft tokens in one weight read, which raises $B$.
• Training and prefill: the weight matmuls are compute-bound. Much of the remaining time goes to memory-bound norms, activations and the attention softmax. These are the targets of fusion (lesson 3) and FlashAttention (lesson 4).
• Diffusion and flow-matching samplers (for example a latent diffusion model of seismic ground motion) are not like LLM decoding: each sampling step runs the network over all latent tokens at once. A DiT-XL (width 1152) on a 1024-token latent has $B = 1024$ tokens per weight read even at batch 1, above $B^* = 606$ for its $1152 \times 1152$ projections: compute-bound. The cost is the number of sampling steps.
• Small MLP surrogates and PINNs of width 512 or less can never be compute-bound in bf16 on an H100: $DF/(D+F) \le 256 < 295$ at any batch size. They are limited by memory traffic and by kernel launch overhead (lesson 3).
• Classic scientific kernels (stencils, sparse matrix–vector products) do less than 1 FLOP per byte. The roofline model was first proposed for such kernels (Williams, Waterman and Patterson, 2009).
• In practice, Nsight Compute shows a roofline chart and a "Speed Of Light" section (compute and memory throughput as a percentage of peak) for each kernel (lesson 13).

7. Common mistakes #

• Using the sparse peak. Product pages list bf16 with 2:4 sparsity (1,979 TFLOP/s on an H100). Dense work gets at most half.
• Using the FLOP/s ratio for a memory-bound op. Elementwise ops and decode steps speed up by the bandwidth ratio, not by the FLOP/s ratio.
• "Big matrices are compute-bound." The intensity of a weight matmul is about $B$, the number of tokens per weight read, not the size of the weight. A $4096 \times 4096$ matmul with 8 tokens is memory-bound.
• Taking $B$ as the number of sequences in training. The $B$ of the theorem counts the tokens that share one weight read: micro-batch × sequence length.
• Expecting batching to help decode attention. Each sequence reads its own KV cache, so the intensity stays about 1 (about $g$ with GQA).
• Reading the roofline as a prediction. It is a lower bound on time: peak rates, perfect overlap, no launch overhead.
• Judging a kernel by its FLOP/s alone. A memory-bound kernel should be judged by its achieved bandwidth.

8. Check yourself #

1. Compute the ridge point of an A100 SXM 80 GB (312 TFLOP/s dense bf16, 2.039 TB/s) and of an H100 in fp8 (1,979 TFLOP/s dense, 3.35 TB/s).
   Answer

   $312/2.039 \approx 153$ FLOP/byte; $1979/3.35 \approx 591$ FLOP/byte.
2. RMSNorm over 16,384 tokens × 4096 features in bf16 on an H100: it reads $x$ once, writes $y$ once and does about 4 FLOPs per element. Give its intensity, regime and minimum time.
   Answer

   $N = 2^{26} = 67.1$ M elements. Bytes $= 2N \cdot 2 = 268$ MB; FLOPs $\approx 4N = 2.7 \times 10^8$; $I \approx 1$ FLOP/byte, memory-bound; $t \ge 268 \times 10^6 / 3.35 \times 10^{12} \approx 80$ µs.
3. Can a bf16 matmul with $D = F = 512$ become compute-bound on an H100 at any batch size? On an A100?
   Answer

   H100: no, because $I < DF/(D+F) = 256 < 295$. A100 ($R = 153$): yes, from $B^* = 153.0/(1 - 2 \cdot 153.0/512) = 380.4$, so $B = 381$.
4. Llama-3-8B has 32 query heads and 8 KV heads ($d_h = 128$). What is the intensity of its decode attention, and does a batch of 64 sequences change it?
   Answer

   $g = 32/8 = 4$, so $I \approx 4$ FLOP/byte: memory-bound. A batch of 64 does not change it, because each sequence brings its own cache: bytes grow with FLOPs.
5. Fused attention at prefill, one head, $d_h = 128$, $T = 256$ tokens, on an H100. Compute- or memory-bound?
   Answer

   $I = T/2 = 128 < 295$: memory-bound. From $T \approx 590$ on (with $I = T/2 \ge 295$) it becomes compute-bound.
6. A 70B-parameter model in bf16 decodes at batch 1 on a (hypothetical) GPU with the bandwidth of an H100 and enough memory for all weights. What is the upper bound on tokens per second?
   Answer

   Weights: $140$ GB. One step reads them once: $140 \times 10^9 / 3.35 \times 10^{12} = 41.8$ ms, so at most about 24 tokens/s. (In practice the model is split over several GPUs, whose bandwidths add up.)

Recall cardsAnswer in your head first, then reveal. Add the cards to your review deck to see them again just before you would forget.

1. Define arithmetic intensity.

   Show answer

2. State the roofline bound on the time of a kernel.

   Show answer

3. What is the ridge point of an H100 SXM in bf16?

   Show answer

4. How many FLOPs does a $(B \times D)(D \times F)$ matmul do?

   Show answer

5. Why is the intensity of a weight matmul about $B$ when $B$ is small?

   Show answer

6. From about which batch size is a bf16 matmul compute-bound on an H100?

   Show answer

7. What is the intensity of decode attention with MHA, and with GQA?

   Show answer

8. A large ReLU moves from an A100 to an H100. What speed-up do you expect, and why?

   Show answer

9. Compute the minimum time of one decode step of a 7B model in bf16 at batch 1 on an H100.

   Show answer

10. Which ops of a transformer are memory-bound in training?

    Show answer

Add these cards to my reviewOpen the review →

9. Interview angle #

• "Why is LLM decoding memory-bound, and what makes it faster?" — Each step reads all weights and the KV cache for few tokens: $I \approx$ batch for the weight matmuls, $\approx 1$ for attention. Batch more sequences, quantize weights and cache, use GQA or speculative decoding.
• "From which batch size is a matmul compute-bound on an H100?" — About peak/bandwidth ≈ 300 tokens in bf16; a little more for finite $D, F$ (345 at $D = F = 4096$).
• "An H100 has 3× the FLOP/s of an A100. Is my inference 3× faster?" — Only the compute-bound parts. Decode gains about the bandwidth ratio, 1.6×.
• "Explain the roofline model." — Achieved FLOP/s $\le \min(P, I \cdot \mathrm{BW})$. Draw the two roofs on log–log axes; the ridge point $P/\mathrm{BW}$ separates memory-bound from compute-bound.
• "Your kernel runs at 20% of peak FLOP/s. Is that bad?" — Compute its intensity first. If it is memory-bound, compare its bandwidth with the HBM peak instead.

10. Practice #

The notebook for this lesson runs in Colab, with NumPy only. It has five exercises, each with a check cell and a folded solution:

1. roofline_time(flops, nbytes, P, BW) and the regime of a kernel.
2. matmul_intensity(B, D, F) and min_compute_bound_batch(D, F, R), checked against the closed form and a brute-force search.
3. Classify every op of one transformer layer for given shapes and phases (prefill, decode).
4. Measure your own computer: count the FLOPs and bytes of x * 2 and A @ B, time them, and plot your CPU's roofline (runs in under 20 seconds).
5. The A100 → H100 speed-up of a list of ops, from their FLOPs and bytes alone.

11. Watch and read #

A very short intro to the Roofline modelNHR@FAU (Erlangen National High Performance Computing Center) · 19 min

The roofline from the people who use it every day in HPC: two hardware numbers, one kernel number, one bound.

Lecture 8: CUDA Performance ChecklistMark Saroufim · GPU MODE lecture 8 · 68 min

Arithmetic intensity, memory-bound versus compute-bound kernels, and what to try for each.

Horace He: Building Machine Learning Systems for a Trillion Trillion Floating Point OperationsHorace He · Jane Street · 63 min

Why ML workloads are limited by compute, memory bandwidth or overhead, from a developer of PyTorch.

• Horace He, Making Deep Learning Go Brrrr From First Principles: compute, bandwidth and overhead, with pictures.
• Austin et al., How to Scale Your Model, All About Rooflines: the same analysis for TPUs and GPUs, with many exercises.
• Williams, Waterman and Patterson (2009), "Roofline: an insightful visual performance model for multicore architectures", CACM; free technical report version.
• NVIDIA, Matrix multiplication background: arithmetic intensity of GEMMs, tile and wave quantization.