# What is the split strategy for large-dimensional reduction operations in triton code generation?

**URL:** <https://dev-discuss.pytorch.org/t/what-is-the-split-strategy-for-large-dimensional-reduction-operations-in-triton-code-generation/1268>\
**Category:** compiler\
**Created:** [May 14, 2023, 5:01pm UTC](https://dev-discuss.pytorch.org/t/what-is-the-split-strategy-for-large-dimensional-reduction-operations-in-triton-code-generation/1268 "2023-05-14T17:01:05Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Minerva\_Yu](https://avatars.discourse-cdn.com/v4/letter/m/7ba0ec/32.png) [@Minerva\_Yu](https://dev-discuss.pytorch.org/u/Minerva_Yu)\
**Post date:** [May 14, 2023, 5:01pm UTC](https://dev-discuss.pytorch.org/t/what-is-the-split-strategy-for-large-dimensional-reduction-operations-in-triton-code-generation/1268/1 "2023-05-14T17:01:05Z")

</div>

For example, if I do max operation:

```auto
def fn(x):
    return torch.max(x, 1)

opt_fn = torch.compile(fn)
x = torch.randn(16, 2**20, device="cuda")
y = opt_fn(x)

```

I found the triton code generated has two kernels to do this reduction op, the first kernel do a smaller dimension reduction, result in for example (16, 2\*\*8) tensor, and the second kernel will do the last reduction and result in (16, 1) tensor. This magic happens in [code](https://github.com/pytorch/pytorch/blob/main/torch/_inductor/ir.py#L544) in ir.py.

But could anyone help me on why it need this strategy? Because of the SM’s capacity or maybe for performance considerations?

Thanks very much!

---

<div class="post-metadata">

**Author:** ![ezyang](https://yyz2.discourse-cdn.com/flex036/user_avatar/dev-discuss.pytorch.org/ezyang/32/12_2.png) [@ezyang](https://dev-discuss.pytorch.org/u/ezyang)\
**Post date:** [May 14, 2023, 5:06pm UTC](https://dev-discuss.pytorch.org/t/what-is-the-split-strategy-for-large-dimensional-reduction-operations-in-triton-code-generation/1268/2 "2023-05-14T17:06:04Z")

</div>

This is split reductions. The goal is to get good utilization, as if you’re reducing on dim=1, the naive strategy is to assign work to SMs based on dim=0, but in your case the tensor only has 16 rows and this wouldn’t fully utilize the GPU. Splitting dim=1 lets us get full utilization at the cost of one small final reduction at the end.