# Question regarding horizontal fusion

**URL:** <https://dev-discuss.pytorch.org/t/question-regarding-horizontal-fusion/3275>\
**Category:** compiler\
**Created:** [December 3, 2025, 10:52pm UTC](https://dev-discuss.pytorch.org/t/question-regarding-horizontal-fusion/3275 "2025-12-03T22:52:46Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Pawl](https://avatars.discourse-cdn.com/v4/letter/p/ea5d25/32.png) [@Pawl](https://dev-discuss.pytorch.org/u/Pawl)\
**Post date:** [December 3, 2025, 10:52pm UTC](https://dev-discuss.pytorch.org/t/question-regarding-horizontal-fusion/3275/1 "2025-12-03T22:52:46Z")

</div>

I am trying to get my head around Inductor. From _inductor/fx_\_passes/ and _inductor/codegen_ it seems like there is support for both vertical and horizontal fusion. By vertical fusion I guess, what is meant is kernel fusion where sequential ops are fused together.

For horizontal fusion, if I understand the design correctly, if you have _op(A,B) and op(A,C)_ and _can\_fuse\_horizontal=True_, then you can do something like _g(op(A, f(A,B)))_ where _f_ and _g_ are op specific transforms that allow you to fuse the op and split the result. Is this correct? Or am I misunderstanding?

And from the code is seems like CUDA and CUTLASS are not yet supported and only Triton is?

So if the codegen defaults to using the CUDA ops then horizontal fusion won’t be possible (for now), but if it chooses Triton than it potentially might be used? Is my understanding correct?

---

<div class="post-metadata">

**Author:** ![jansel](https://yyz2.discourse-cdn.com/flex036/user_avatar/dev-discuss.pytorch.org/jansel/32/74_2.png) [@jansel](https://dev-discuss.pytorch.org/u/jansel)\
**Post date:** [December 9, 2025, 9:22pm UTC](https://dev-discuss.pytorch.org/t/question-regarding-horizontal-fusion/3275/2 "2025-12-09T21:22:52Z")

</div>

Both horizontal and vertical fusion is supported for pointwise and reduction ops when the shapes are compatible. This is true for the Triton, C++, Halide, and Metal backends on all device types. This is done in the inductor scheduler.

There are heuristics to decide when these fusions are profitable, some of which can be controlled via configs such as:

> <https://github.com/pytorch/pytorch/blob/06b03073b5fe2fb26260323a69a9d5ab23bbc9cd/torch/_inductor/config.py#L716-L717>

and:

> <https://github.com/pytorch/pytorch/blob/06b03073b5fe2fb26260323a69a9d5ab23bbc9cd/torch/_inductor/config.py#L1534-L1536>

and many more (search that file for fusion).

For matmul/conv/etc we support epilogue fusion when in max-autotune mode and the Triton/CUTLASS template is selected. Plus prologue fusion in some quantization case.

---

<div class="post-metadata">

**Author:** ![Pawl](https://avatars.discourse-cdn.com/v4/letter/p/ea5d25/32.png) [@Pawl](https://dev-discuss.pytorch.org/u/Pawl)\
**Post date:** [December 10, 2025, 1:48pm UTC](https://dev-discuss.pytorch.org/t/question-regarding-horizontal-fusion/3275/3 "2025-12-10T13:48:17Z")

</div>

Thank you, this answers all my questions.

Out of curiosity I also searched for the roadmap and if I’m understanding correctly, horizontal fusion of matmuls is a pending item. Very cool.

> `KR3.2 Explicit API for horizontal fusion, foreach_map, that includes grouped gemm. SOTA perf on grouped linear MOE.`

> **[\[PUBLIC\] PyTorch Compiler Core - H1 2025 Roadmap.pdf](https://drive.google.com/file/d/1WthsinddTwcvS2eXffnH_xbDAI-lzewQ/view?usp=drive_link)**
>
> Google Drive file.