# Ufunc codegen for pointwise operators

**URL:** https://dev-discuss.pytorch.org/t/ufunc-codegen-for-pointwise-operators/367
**Category:** Uncategorized
**Created:** [October 6, 2021, 9:15pm UTC](https://dev-discuss.pytorch.org/t/ufunc-codegen-for-pointwise-operators/367 "2021-10-06T21:15:54Z")
**Posts on this page:** 1
**Page:** 1

<div class="post-metadata">

### Author: ![ezyang](https://yyz2.discourse-cdn.com/flex036/user_avatar/dev-discuss.pytorch.org/ezyang/32/12_2.png) [@ezyang](https://dev-discuss.pytorch.org/u/ezyang)
#### Post date: [October 6, 2021, 9:15pm UTC](https://dev-discuss.pytorch.org/t/ufunc-codegen-for-pointwise-operators/367/1 "2021-10-06T21:15:54Z")

</div>

Our goal is to define every ufunc in PyTorch as a simple templated inner loop function like below:

```auto
template <typename T>
C10_HOST_DEVICE T add(T self, T other, T alpha) {
 return self + alpha * other;
}

```

and transform this into a full torch.add kernel.

Proposal for how to do this is at [Ufunc codegen for pointwise operators - Google Docs](https://docs.google.com/document/d/12rtlHnPUpaJ-I52Iob3L0WA3rKRr_OY7fXqeCvn2MVY/edit#)  
WIP PR is [ufunc codegen by ezyang · Pull Request #65851 · pytorch/pytorch · GitHub](https://github.com/pytorch/pytorch/pull/65851)