Decoding: f32 vs int4 weights
One token through Whisper-tiny's decoder layer shapes. GPU time from timestamp queries, median of several runs. "Weights GB/s" counts the bytes each format actually reads. Output error compares the int4 result to the f32 result for the same random weights.
| shape (M×N×K) | what | format | ms | weights MB | weights GB/s | speedup | output rel. error |
|---|