# Review request: custom-kernel attribution benchmarks vs torch.compile for decode workloads

**URL:** <https://dev-discuss.pytorch.org/t/review-request-custom-kernel-attribution-benchmarks-vs-torch-compile-for-decode-workloads/3411>\
**Category:** compiler\
**Created:** [July 1, 2026, 5:54am UTC](https://dev-discuss.pytorch.org/t/review-request-custom-kernel-attribution-benchmarks-vs-torch-compile-for-decode-workloads/3411 "2026-07-01T05:54:04Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![MilkClouds](https://yyz2.discourse-cdn.com/flex036/user_avatar/dev-discuss.pytorch.org/milkclouds/32/3286_2.png) [@MilkClouds](https://dev-discuss.pytorch.org/u/MilkClouds)\
**Post date:** [July 1, 2026, 5:54am UTC](https://dev-discuss.pytorch.org/t/review-request-custom-kernel-attribution-benchmarks-vs-torch-compile-for-decode-workloads/3411/1 "2026-07-01T05:54:04Z")

</div>

Hi PyTorch compiler folks,

I put together a small benchmark repo on custom-kernel attribution for decode/generation workloads:

> **[GitHub - MilkClouds/do-custom-kernels-matter: Measure whether custom kernels materially improve...](https://github.com/MilkClouds/do-custom-kernels-matter)**
>
> Measure whether custom kernels materially improve end-to-end inference beyond static-cache, CUDA-graph, and torch.compile baselines.

The narrow claim is not that custom kernels are useless. The claim is that in these public-style inference optimization repos, the large end-to-end gains often come from static cache / graph capture / batching, while the marginal handwritten Triton/CUDA kernel contribution is much smaller. In the two reproduced case studies, a correctly configured `torch.compile` baseline matches or beats the custom-kernel path.

I would appreciate a review from the PyTorch compiler side on:

1. Whether the `torch.compile` baselines are configured fairly.

2. Whether the Qwen3.5 and Qwen3-TTS compile boundaries are reasonable.

3. Whether the benchmark framing around `StaticCache`, CUDA graphs, and decode workloads is technically accurate.

4. Any missing PyTorch compiler features/docs that would make this kind of baseline easier for users to discover.

I am not looking for endorsement of the conclusion; I would most value critical feedback on the benchmark methodology.