# compiler **URL:** https://dev-discuss.pytorch.org/c/compiler/5.md?page=2 [Latest](https://dev-discuss.pytorch.org/latest.md) · [Categories](https://dev-discuss.pytorch.org/categories.md) **Page:** 3 --- ## [Defining the Core ATen Opset](https://dev-discuss.pytorch.org/t/defining-the-core-aten-opset/1464)
**Author:** [@SS-JIA](https://dev-discuss.pytorch.org/u/SS-JIA)\ **Replies:** 12\ **Last updated:** [August 21, 2024, 3:51am UTC](https://dev-discuss.pytorch.org/t/defining-the-core-aten-opset/1464 "2024-08-21T03:51:02Z")
TL;DR Folks from across Meta internal PyTorch Core, PyTorch Edge, and PyTorch Compiler teams collaborated to review a list of commonly used ATen operators and discussed whether each should be added to the core ATen opera… --- ## [Why core aten IR doesn’t contain scalar\_tensor overload version of some ops, like bitwise-like ops, remainder and so on?](https://dev-discuss.pytorch.org/t/why-core-aten-ir-doesn-t-contain-scalar-tensor-overload-version-of-some-ops-like-bitwise-like-ops-remainder-and-so-on/2398)
**Author:** [@lingzhi98](https://dev-discuss.pytorch.org/u/lingzhi98)\ **Replies:** 0\ **Last updated:** [August 21, 2024, 2:24am UTC](https://dev-discuss.pytorch.org/t/why-core-aten-ir-doesn-t-contain-scalar-tensor-overload-version-of-some-ops-like-bitwise-like-ops-remainder-and-so-on/2398 "2024-08-21T02:24:49Z")
Many ops has scalar\_tensor version in native\_functions.yaml, but are not included in core aten IR set. Some of them has same results between scalar\_tensor version and scalar version like bitwise\_and, but some are differe… --- ## [How to Access Triton Kernels from TorchInductor when running on CPU?](https://dev-discuss.pytorch.org/t/how-to-access-triton-kernels-from-torchinductor-when-running-on-cpu/2370)
**Author:** [@JoeLi12345](https://dev-discuss.pytorch.org/u/JoeLi12345)\ **Replies:** 1\ **Last updated:** [August 12, 2024, 1:39am UTC](https://dev-discuss.pytorch.org/t/how-to-access-triton-kernels-from-torchinductor-when-running-on-cpu/2370 "2024-08-12T01:39:33Z")
When running torch.compile for a PyTorch model and accessing the generated TorchInductor python code for graph with C++/Triton kernels as indicated here, there appears to be no way to access the generated Triton code whe… --- ## [Connecting PyTorch sparse tensors with MLIR](https://dev-discuss.pytorch.org/t/connecting-pytorch-sparse-tensors-with-mlir/1803)
**Author:** [@aartbik](https://dev-discuss.pytorch.org/u/aartbik)\ **Replies:** 4\ **Last updated:** [August 8, 2024, 6:01pm UTC](https://dev-discuss.pytorch.org/t/connecting-pytorch-sparse-tensors-with-mlir/1803 "2024-08-08T18:01:06Z")
Greetings! First time poster in this forum, after incorrectly starting the topic in the user forum. I won’t repeat those postings here (since I hope you will read them there), but in a nutshell, the MLIR Sparsifier team… --- ## [JIT scripting & Autocast](https://dev-discuss.pytorch.org/t/jit-scripting-autocast/139)
**Author:** [@lemo](https://dev-discuss.pytorch.org/u/lemo)\ **Replies:** 12\ **Last updated:** [August 8, 2024, 4:56am UTC](https://dev-discuss.pytorch.org/t/jit-scripting-autocast/139 "2024-08-08T04:56:14Z")
Autocast (aka Automatic Mixed Precision) is an optimization which helps taking advantage of the storage and performance benefits of narrow types (float16) while preserving the additional range and numerical precision of … --- ## [PyTorch/XLA 2.4 dev update](https://dev-discuss.pytorch.org/t/pytorch-xla-2-4-dev-update/2356)
**Author:** [@JackCaoG](https://dev-discuss.pytorch.org/u/JackCaoG)\ **Replies:** 0\ **Last updated:** [August 6, 2024, 6:05pm UTC](https://dev-discuss.pytorch.org/t/pytorch-xla-2-4-dev-update/2356 "2024-08-06T18:05:12Z")
PyTorch/XLA 2.4 Dev Update Hey I am here to give a late update for the PyTorch/XLA 2.3 release. Similar to my previous update, you can check our release note for detailed updates. I am going to highlight some of the new … --- ## [How to get the backward graph while using torch.export?](https://dev-discuss.pytorch.org/t/how-to-get-the-backward-graph-while-using-torch-export/2318)
**Author:** [@fishmingyu](https://dev-discuss.pytorch.org/u/fishmingyu)\ **Replies:** 4\ **Last updated:** [July 29, 2024, 10:42pm UTC](https://dev-discuss.pytorch.org/t/how-to-get-the-backward-graph-while-using-torch-export/2318 "2024-07-29T22:42:22Z")
I have read the references such as topic-1621 and aot\_autograd. However, I am unable to find a solution that enables exporting the backward graph while using torch.export. Is there any trick to do this? I provide a simpl… --- ## [TorchInductor: a PyTorch-native Compiler with Define-by-Run IR and Symbolic Shapes](https://dev-discuss.pytorch.org/t/torchinductor-a-pytorch-native-compiler-with-define-by-run-ir-and-symbolic-shapes/747)
**Author:** [@jansel](https://dev-discuss.pytorch.org/u/jansel)\ **Replies:** 46\ **Last updated:** [July 29, 2024, 8:09am UTC](https://dev-discuss.pytorch.org/t/torchinductor-a-pytorch-native-compiler-with-define-by-run-ir-and-symbolic-shapes/747 "2024-07-29T08:09:49Z")
The PyTorch team has been building TorchDynamo, which helps to solve the graph capture problem of PyTorch with dynamic Python bytecode transformation. To actually make PyTorch faster, TorchDynamo must be paired with a co… --- ## [How to set wrap function using TorchDynamo graph capture?](https://dev-discuss.pytorch.org/t/how-to-set-wrap-function-using-torchdynamo-graph-capture/2185)
**Author:** [@wmhst7](https://dev-discuss.pytorch.org/u/wmhst7)\ **Replies:** 4\ **Last updated:** [June 28, 2024, 10:33pm UTC](https://dev-discuss.pytorch.org/t/how-to-set-wrap-function-using-torchdynamo-graph-capture/2185 "2024-06-28T22:33:07Z")
I am trying to capture both forward and backward graphs using: m, \_ = aot\_export\_module(fwd, \[inp\], trace\_joint=True, output\_loss\_index=0, decompositions=None) fwd, bwd = default\_partition(m, \[inp\], num\_fwd\_outputs=1) A… --- ## [Understanding torch.fx.traceback.preserve\_node\_meta()](https://dev-discuss.pytorch.org/t/understanding-torch-fx-traceback-preserve-node-meta/2321)
**Author:** [@TharinduRusira](https://dev-discuss.pytorch.org/u/TharinduRusira)\ **Replies:** 0\ **Last updated:** [July 26, 2024, 8:04pm UTC](https://dev-discuss.pytorch.org/t/understanding-torch-fx-traceback-preserve-node-meta/2321 "2024-07-26T20:04:36Z")
I previously posted this topic on PyTorch Forums but I think this is a topic best suited for the dev forum. I’m trying to understand the scope of torch.fx.traceback.preserve\_node\_meta(). Can someone please explain the s… --- ## [Supporting Dynamo in Python 3.12](https://dev-discuss.pytorch.org/t/supporting-dynamo-in-python-3-12/2320)
**Author:** [@williamwen42](https://dev-discuss.pytorch.org/u/williamwen42)\ **Replies:** 0\ **Last updated:** [July 26, 2024, 6:41pm UTC](https://dev-discuss.pytorch.org/t/supporting-dynamo-in-python-3-12/2320 "2024-07-26T18:41:21Z")
With another year comes a new Python version for us to support! Fortunately, enabling Python 3.12 support in Dynamo was not as challenging as supporting Python 3.11 (you can read our technical blog posts on supporting Py… --- ## [Inlining a custom triton kernel](https://dev-discuss.pytorch.org/t/inlining-a-custom-triton-kernel/2305)
**Author:** [@fhossein-quic](https://dev-discuss.pytorch.org/u/fhossein-quic)\ **Replies:** 0\ **Last updated:** [July 22, 2024, 7:55pm UTC](https://dev-discuss.pytorch.org/t/inlining-a-custom-triton-kernel/2305 "2024-07-22T19:55:09Z")
Hi all, I have a custom backend, and can successfully generate triton codes torch models via torchInductor and run them on my tritn backend. I am trying to incorporate user-defined triton kernels by following the exampl… --- ## [Compiled autograd with custom ops error](https://dev-discuss.pytorch.org/t/compiled-autograd-with-custom-ops-error/2295)
**Author:** [@tdomagal](https://dev-discuss.pytorch.org/u/tdomagal)\ **Replies:** 1\ **Last updated:** [July 19, 2024, 8:55am UTC](https://dev-discuss.pytorch.org/t/compiled-autograd-with-custom-ops-error/2295 "2024-07-19T08:55:01Z")
I try to use custom op implementation with enabled compiled autograd and enabled autocast. My op forward and backward are decorated with torch.cuda.amp decorators: custom\_bwd, custom\_fwd Receiving the following error: … --- ## [AOTAutograd incorrect lowering composite ops in inference\_mode](https://dev-discuss.pytorch.org/t/aotautograd-incorrect-lowering-composite-ops-in-inference-mode/2275)
**Author:** [@wdziurdz](https://dev-discuss.pytorch.org/u/wdziurdz)\ **Replies:** 3\ **Last updated:** [July 17, 2024, 2:36pm UTC](https://dev-discuss.pytorch.org/t/aotautograd-incorrect-lowering-composite-ops-in-inference-mode/2275 "2024-07-17T14:36:51Z")
I have a problem when using aot\_autograd with my own backend and torch.compile in in inference mode. Example code: def test1(): with torch.inference\_mode(True): @torch.compile(backend='my\_backend') … --- ## [\[RFC\] Performance profiling at scale with detailed NVTX annotations](https://dev-discuss.pytorch.org/t/rfc-performance-profiling-at-scale-with-detailed-nvtx-annotations/2224)
**Author:** [@AlexDenisov](https://dev-discuss.pytorch.org/u/AlexDenisov)\ **Replies:** 0\ **Last updated:** [July 10, 2024, 10:05am UTC](https://dev-discuss.pytorch.org/t/rfc-performance-profiling-at-scale-with-detailed-nvtx-annotations/2224 "2024-07-10T10:05:11Z")
Hi folks, I’ve been looking at nsys traces for end2end training runs and at some point, as the trace size grows, it becomes infeasible to use Nsight Systems. So I hacked together a simple solution based on NVTX annotat… --- ## [State of symbolic shapes branch](https://dev-discuss.pytorch.org/t/state-of-symbolic-shapes-branch/777)
**Author:** [@ezyang](https://dev-discuss.pytorch.org/u/ezyang)\ **Replies:** 96\ **Last updated:** [July 7, 2024, 2:04am UTC](https://dev-discuss.pytorch.org/t/state-of-symbolic-shapes-branch/777 "2024-07-07T02:04:21Z")
State of symbolic-shapes branch: Sep 17 edition The symbolic-shapes branch (PyTorch: https://github.com/pytorch/pytorch/pull/84246 ; torchdynamo: \[WIP branch\] symbolic shape hacking by Chillee · Pull Request #1180 · pyto… --- ## [Why PyTorch does not need a new standardized operator set](https://dev-discuss.pytorch.org/t/why-pytorch-does-not-need-a-new-standardized-operator-set/2206)
**Author:** [@jansel](https://dev-discuss.pytorch.org/u/jansel)\ **Replies:** 3\ **Last updated:** [July 3, 2024, 11:33pm UTC](https://dev-discuss.pytorch.org/t/why-pytorch-does-not-need-a-new-standardized-operator-set/2206 "2024-07-03T23:33:58Z")
with @gchanan Background: There is no right answer only trade-offs There have been many past efforts to build a standardized operator sets for PyTorch: PrimTorch was the most expansive effort, taking a maximal positio… --- ## [Understanding the lifecycle of compiled code cache entry](https://dev-discuss.pytorch.org/t/understanding-the-lifecycle-of-compiled-code-cache-entry/2205)
**Author:** [@youkaichao](https://dev-discuss.pytorch.org/u/youkaichao)\ **Replies:** 2\ **Last updated:** [July 2, 2024, 9:31pm UTC](https://dev-discuss.pytorch.org/t/understanding-the-lifecycle-of-compiled-code-cache-entry/2205 "2024-07-02T21:31:11Z")
I hit a corner case, where the compiled code cache entry is gone after function returns. So I want to ask a general question here, when can we expect a compiled code cache entry will be removed? And how is this achieved… --- ## [Semantics of None in aten.index](https://dev-discuss.pytorch.org/t/semantics-of-none-in-aten-index/2203)
**Author:** [@anieto](https://dev-discuss.pytorch.org/u/anieto)\ **Replies:** 1\ **Last updated:** [July 2, 2024, 4:53pm UTC](https://dev-discuss.pytorch.org/t/semantics-of-none-in-aten-index/2203 "2024-07-02T16:53:07Z")
What is the meaning of None indices as arguments to core ATen’s aten.index, which has type signature: index.Tensor(Tensor self, Tensor?\[\] indices) -\> Tensor One guess is that a None index stands for the entire correspo… --- ## [Fixing torch.compile Reference Leaks (Automatic Deletion of Dynamo Code Objects)](https://dev-discuss.pytorch.org/t/fixing-torch-compile-reference-leaks-automatic-deletion-of-dynamo-code-objects/2197)
**Author:** [@williamwen42](https://dev-discuss.pytorch.org/u/williamwen42)\ **Replies:** 0\ **Last updated:** [July 2, 2024, 7:27am UTC](https://dev-discuss.pytorch.org/t/fixing-torch-compile-reference-leaks-automatic-deletion-of-dynamo-code-objects/2197 "2024-07-02T07:27:00Z")
We recently fixed several torch.compile refleaks (reference leaks) as a part of improving Dynamo reliability. Refleaks were reported in issues such as CUDA Memory leak w/ torch.compile in both stable and trunk · Issue #1… --- ## [Reverse Fusion of Node Pairs in Scheduler](https://dev-discuss.pytorch.org/t/reverse-fusion-of-node-pairs-in-scheduler/2153)
**Author:** [@NiklasZ](https://dev-discuss.pytorch.org/u/NiklasZ)\ **Replies:** 0\ **Last updated:** [June 14, 2024, 10:15am UTC](https://dev-discuss.pytorch.org/t/reverse-fusion-of-node-pairs-in-scheduler/2153 "2024-06-14T10:15:59Z")
Hi, I’m interested in PyTorch’s operator fusion. I have been reading the Scheduler’s source code and noticed a strange case where it seems to consider the fusability of a pair of nodes node1, node2 in the normal order: s… --- ## [Custom C++ External Kernel for TorchInductor](https://dev-discuss.pytorch.org/t/custom-c-external-kernel-for-torchinductor/2140)
**Author:** [@fhossein-quic](https://dev-discuss.pytorch.org/u/fhossein-quic)\ **Replies:** 2\ **Last updated:** [June 10, 2024, 10:29am UTC](https://dev-discuss.pytorch.org/t/custom-c-external-kernel-for-torchinductor/2140 "2024-06-10T10:29:45Z")
Is there a way to use a custom C++ lowering of a torch op (which is not part of aten) as an external kernel in TorchInductor? In TorchInductor, we can choose to lower a torch op either with external kernels or triton ke… --- ## [PyTorch/XLA 2.3 dev update](https://dev-discuss.pytorch.org/t/pytorch-xla-2-3-dev-update/2124)
**Author:** [@JackCaoG](https://dev-discuss.pytorch.org/u/JackCaoG)\ **Replies:** 0\ **Last updated:** [May 29, 2024, 4:19am UTC](https://dev-discuss.pytorch.org/t/pytorch-xla-2-3-dev-update/2124 "2024-05-29T04:19:31Z")
Hey I am here to give a late update for the PyTorch/XLA 2.3 release. Similar to my previous update, you can check our release note for detailed updates. I am going to highlight some of the new features and share how I th… --- ## [User-defined Kernels vs. \`torch.library\` custom op](https://dev-discuss.pytorch.org/t/user-defined-kernels-vs-torch-library-custom-op/2113)
**Author:** [@jeromeku](https://dev-discuss.pytorch.org/u/jeromeku)\ **Replies:** 1\ **Last updated:** [May 22, 2024, 4:47pm UTC](https://dev-discuss.pytorch.org/t/user-defined-kernels-vs-torch-library-custom-op/2113 "2024-05-22T16:47:11Z")
If I have a user-defined triton kernel as part of a torch.nn.Module, what is the difference between: Registering the user-defined triton kernel using the torch.library API and compiling the module vs. Calling torch.c… --- ## [Different points at which fusion occurs?](https://dev-discuss.pytorch.org/t/different-points-at-which-fusion-occurs/2099)
**Author:** [@Amogh](https://dev-discuss.pytorch.org/u/Amogh)\ **Replies:** 8\ **Last updated:** [May 21, 2024, 10:26pm UTC](https://dev-discuss.pytorch.org/t/different-points-at-which-fusion-occurs/2099 "2024-05-21T22:26:04Z")
I understand that can\_fuse and scores\_fusion are responsible for fusion of buffers at the Inductor IR level. But I also see some fusions in the fx\_passes before the lowering to Inductor IR goes through. Can someone clari… --- ## [PyTorch Sparse(GNN) Compiler RFC](https://dev-discuss.pytorch.org/t/pytorch-sparse-gnn-compiler-rfc/1644)
**Author:** [@fishmingyu](https://dev-discuss.pytorch.org/u/fishmingyu)\ **Replies:** 28\ **Last updated:** [May 21, 2024, 9:00am UTC](https://dev-discuss.pytorch.org/t/pytorch-sparse-gnn-compiler-rfc/1644 "2024-05-21T09:00:10Z")
:rocket: The feature, motivation and pitch Torch.compile has been included in Pytorch 2.0 or late versions and has shown that it is a powerful way to speed up Pytorch code greatly. Pytorch inductor has pioneered graph-mo… --- ## [How to replace a callable with an opaque python callable in an Inductor pass?](https://dev-discuss.pytorch.org/t/how-to-replace-a-callable-with-an-opaque-python-callable-in-an-inductor-pass/2082)
**Author:** [@sanchit](https://dev-discuss.pytorch.org/u/sanchit)\ **Replies:** 6\ **Last updated:** [May 15, 2024, 8:49pm UTC](https://dev-discuss.pytorch.org/t/how-to-replace-a-callable-with-an-opaque-python-callable-in-an-inductor-pass/2082 "2024-05-15T20:49:49Z")
Hi, HigherOrderOperators seem to be helpful for tracing models with opaque python code during a torch.compile invocation. But can some HigherOrderOperator that uses some arbitrary python be used to replace a Node’s ca… --- ## [How To Bring Compile Time Down to Zero: Our Plans and Direction (May 14th Edition)](https://dev-discuss.pytorch.org/t/how-to-bring-compile-time-down-to-zero-our-plans-and-direction-may-14th-edition/2089)
**Author:** [@oulgen](https://dev-discuss.pytorch.org/u/oulgen)\ **Replies:** 0\ **Last updated:** [May 15, 2024, 4:45pm UTC](https://dev-discuss.pytorch.org/t/how-to-bring-compile-time-down-to-zero-our-plans-and-direction-may-14th-edition/2089 "2024-05-15T16:45:15Z")
We are excited to announce that over the course of the first half of 2024 we have been prioritizing improving compile times for torch.compile workflows. Swift iterations and efficient development cycles are very importan… --- ## [When does the inductor code run?](https://dev-discuss.pytorch.org/t/when-does-the-inductor-code-run/2088)
**Author:** [@whyzzy](https://dev-discuss.pytorch.org/u/whyzzy)\ **Replies:** 5\ **Last updated:** [May 15, 2024, 9:40am UTC](https://dev-discuss.pytorch.org/t/when-does-the-inductor-code-run/2088 "2024-05-15T09:40:50Z")
Hello, I am new to torch.compile and want to study the inductor. I wonder when does the inductor’s optimization code run? for example, there are main.py code here. def f(x): return torch.sin(x)\*\*2 + torch.cos(x)\*\*… --- ## [What is the mental model for torch.compile to deal with stateful objects in general?](https://dev-discuss.pytorch.org/t/what-is-the-mental-model-for-torch-compile-to-deal-with-stateful-objects-in-general/2080)
**Author:** [@youkaichao](https://dev-discuss.pytorch.org/u/youkaichao)\ **Replies:** 3\ **Last updated:** [May 13, 2024, 8:01pm UTC](https://dev-discuss.pytorch.org/t/what-is-the-mental-model-for-torch-compile-to-deal-with-stateful-objects-in-general/2080 "2024-05-13T20:01:06Z")
I’m trying to get a clear mental model for torch.compile to deal with stateful objects. Traditionally, JIT compilers such as numba only compile for arrays and several Python basic data structures. It is easy to figure o… [Previous page](https://dev-discuss.pytorch.org/c/compiler/5.md?page=1) [Next page](https://dev-discuss.pytorch.org/c/compiler/5.md?page=3)