Attention: fused vs unfused
Whisper-tiny attention shapes (6 heads, head dim 64). Unfused = QKᵀ, softmax, PV as three kernels (time is their sum); fused = one FlashAttention-style kernel. GPU timestamps, median of several runs. "Intermediate" is the score + probability matrices the unfused version writes.
| shape (H×Tq×Tk) | what | unfused ms | fused ms | speedup | intermediate avoided |
|---|