I compared autoregressive decoding with two-token MTP in a controlled vLLM deployment and observed a 1.91x to 2.19x increase in output throughput. The dominant MTP GEMM was not faster than the dominant AR GEMV. Instead, MTP made greater token progress while requiring substantially fewer recurring GPU executions per generated token.
I followed this result through runtime acceptance telemetry, PyTorch Profiler, NVIDIA Nsight Systems, and Nsight Compute. I am sharing the investigation both as a performance report and to request feedback on the interpretation of the PyTorch runtime hierarchy and its relationship to recurring CUDA Graph executions.
Full preprint: