# compiler **URL:** https://dev-discuss.pytorch.org/c/compiler/5.md?page=3 [Latest](https://dev-discuss.pytorch.org/latest.md) · [Categories](https://dev-discuss.pytorch.org/categories.md) **Page:** 4 --- ## [How can I dump the prims IR, triton code, and ptx code when using torch.compile()](https://dev-discuss.pytorch.org/t/how-can-i-dump-the-prims-ir-triton-code-and-ptx-code-when-using-torch-compile/2057)
**Author:** [@anneouyang](https://dev-discuss.pytorch.org/u/anneouyang)\ **Replies:** 2\ **Last updated:** [May 6, 2024, 9:45am UTC](https://dev-discuss.pytorch.org/t/how-can-i-dump-the-prims-ir-triton-code-and-ptx-code-when-using-torch-compile/2057 "2024-05-06T09:45:04Z")
Is there a way to get the code at each of the stages? --- ## [Torch.compile support for Python 3.12 completed](https://dev-discuss.pytorch.org/t/torch-compile-support-for-python-3-12-completed/2054)
**Author:** [@williamwen42](https://dev-discuss.pytorch.org/u/williamwen42)\ **Replies:** 0\ **Last updated:** [May 3, 2024, 5:38pm UTC](https://dev-discuss.pytorch.org/t/torch-compile-support-for-python-3-12-completed/2054 "2024-05-03T17:38:12Z")
Signal boosting that Python 3.12 support has been added to torch.compile and has been present in the nightly builds for a while. We anticipate that this feature will be included in the PyTorch 2.4 release. We encountere… --- ## [Performance Comparison between Torch.Compile and APEX optimizers](https://dev-discuss.pytorch.org/t/performance-comparison-between-torch-compile-and-apex-optimizers/2023)
**Author:** [@mlazos](https://dev-discuss.pytorch.org/u/mlazos)\ **Replies:** 1\ **Last updated:** [May 1, 2024, 11:39pm UTC](https://dev-discuss.pytorch.org/t/performance-comparison-between-torch-compile-and-apex-optimizers/2023 "2024-05-01T23:39:43Z")
TL;DR Compiled Adam outperformed SOTA hand-optimized APEX optimizers on all benchmarks; 62.99% on Torchbench, 53.18% on HuggingFace, 142.75% on TIMM and 88.13% on BlueBerries Compiled AdamW performed similarly with up t… --- ## [State of PT2 OSS Issues: Q1 2024](https://dev-discuss.pytorch.org/t/state-of-pt2-oss-issues-q1-2024/1998)
**Author:** [@penguinwu](https://dev-discuss.pytorch.org/u/penguinwu)\ **Replies:** 0\ **Last updated:** [April 10, 2024, 11:36am UTC](https://dev-discuss.pytorch.org/t/state-of-pt2-oss-issues-q1-2024/1998 "2024-04-10T11:36:47Z")
Criteria of success 90% of PT2 high-priority issues closed (H1 roadmap KR) 75% of PT2 bug issues are closed (H1 BE KR) “Unclassified” open PT2 issues 0% “unclassified” open PT2 high-priority issues opened more than a … --- ## [Inductor CUDA Backend](https://dev-discuss.pytorch.org/t/inductor-cuda-backend/1689)
**Author:** [@jeromeku](https://dev-discuss.pytorch.org/u/jeromeku)\ **Replies:** 1\ **Last updated:** [April 4, 2024, 10:35am UTC](https://dev-discuss.pytorch.org/t/inductor-cuda-backend/1689 "2024-04-04T10:35:56Z")
Is there any documentation on the different CUDA kernel backends that inductor selects from (cutlass, triton, etc.)? More specifically, for each backend would like to understand how modules / layers / ops are mapped to … --- ## [TorchDynamo Update 9: Making DDP Work with TorchDynamo](https://dev-discuss.pytorch.org/t/torchdynamo-update-9-making-ddp-work-with-torchdynamo/860)
**Author:** [@davidberard98](https://dev-discuss.pytorch.org/u/davidberard98)\ **Replies:** 8\ **Last updated:** [November 27, 2023, 8:54pm UTC](https://dev-discuss.pytorch.org/t/torchdynamo-update-9-making-ddp-work-with-torchdynamo/860 "2023-11-27T20:54:40Z")
TL;DR: Previously, torchdynamo interrupted compute-communication overlap in DDP to a sufficient degree that DDP training with dynamo was up to 25% slower than DDP training with eager. We modified dynamo to add additional… --- ## [Difference torch dynamo and torch script](https://dev-discuss.pytorch.org/t/difference-torch-dynamo-and-torch-script/1973)
**Author:** [@Aruna](https://dev-discuss.pytorch.org/u/Aruna)\ **Replies:** 0\ **Last updated:** [March 27, 2024, 4:37am UTC](https://dev-discuss.pytorch.org/t/difference-torch-dynamo-and-torch-script/1973 "2024-03-27T04:37:44Z")
Hi As torch dynamo backends provide fusion and also there is fusions available with torchscript apis. What are the difference between these two? Thank you --- ## [A TorchDynamo trace time ablation study](https://dev-discuss.pytorch.org/t/a-torchdynamo-trace-time-ablation-study/1961)
**Author:** [@jansel](https://dev-discuss.pytorch.org/u/jansel)\ **Replies:** 0\ **Last updated:** [March 22, 2024, 12:47am UTC](https://dev-discuss.pytorch.org/t/a-torchdynamo-trace-time-ablation-study/1961 "2024-03-22T00:47:21Z")
TorchDynamo compile times have been a focus lately. In this post I wanted to give a breakdown of where they are coming from. I will show some profile results and perform an ablation study where I try to estimate how mu… --- ## [Supporting mutations in torch.export.export](https://dev-discuss.pytorch.org/t/supporting-mutations-in-torch-export-export/1470)
**Author:** [@qihqi](https://dev-discuss.pytorch.org/u/qihqi)\ **Replies:** 2\ **Last updated:** [March 20, 2024, 4:07pm UTC](https://dev-discuss.pytorch.org/t/supporting-mutations-in-torch-export-export/1470 "2024-03-20T16:07:01Z")
If a module mutates it’s buffers in it’s forward, on torch.export.export; one would expect it to functionalize the graph and returns a ExportedProgram with functional graph along with extra return values (as specified in… --- ## [Meaning of strict=False in torch.export.export](https://dev-discuss.pytorch.org/t/meaning-of-strict-false-in-torch-export-export/1952)
**Author:** [@anieto](https://dev-discuss.pytorch.org/u/anieto)\ **Replies:** 0\ **Last updated:** [March 20, 2024, 4:01pm UTC](https://dev-discuss.pytorch.org/t/meaning-of-strict-false-in-torch-export-export/1952 "2024-03-20T16:01:40Z")
Pytorch 2.2 adds a strict=False option to torch.export that is thus described in the docs: strict (bool) – When enabled (default), the export function will trace the program through TorchDynamo which will ensure the so… --- ## [Export sub-graphs at the Aten IR level](https://dev-discuss.pytorch.org/t/export-sub-graphs-at-the-aten-ir-level/1936)
**Author:** [@rawwds](https://dev-discuss.pytorch.org/u/rawwds)\ **Replies:** 2\ **Last updated:** [March 14, 2024, 3:29am UTC](https://dev-discuss.pytorch.org/t/export-sub-graphs-at-the-aten-ir-level/1936 "2024-03-14T03:29:09Z")
Regarding the torch.export documentation, it is designed to trace the full graph at the Aten IR level, but potential failures in some corner cases. In such instances, is it possible to export multiple subgraphs, at the A… --- ## [PyTorch 2.0 User Empathy Day Recap](https://dev-discuss.pytorch.org/t/pytorch-2-0-user-empathy-day-recap/1932)
**Author:** [@yanboliang](https://dev-discuss.pytorch.org/u/yanboliang)\ **Replies:** 2\ **Last updated:** [March 13, 2024, 5:49am UTC](https://dev-discuss.pytorch.org/t/pytorch-2-0-user-empathy-day-recap/1932 "2024-03-13T05:49:32Z")
The rules and the outputs We hosted the first PyTorch 2.0 User Empathy Day at Meta, where people picked up trending OSS models, applied torch.compile as a USER and gave it as many fixes as possible without torching PT2 i… --- ## [Torch.compile() + FSDP - Dec 8th](https://dev-discuss.pytorch.org/t/torch-compile-fsdp-dec-8th/1718)
**Author:** [@voz](https://dev-discuss.pytorch.org/u/voz)\ **Replies:** 2\ **Last updated:** [March 8, 2024, 9:39am UTC](https://dev-discuss.pytorch.org/t/torch-compile-fsdp-dec-8th/1718 "2024-03-08T09:39:09Z")
Most of our updates on compiling FSDP have been internal. This is the first public post in this series. Reflection What started out as a project around ~May-June is slowly getting to a place where the changes left to m… --- ## [TorchInductor Update 7: key optimizations with CPU backend in PyTorch 2.2 release](https://dev-discuss.pytorch.org/t/torchinductor-update-7-key-optimizations-with-cpu-backend-in-pytorch-2-2-release/1867)
**Author:** [@leslie-fang-intel](https://dev-discuss.pytorch.org/u/leslie-fang-intel)\ **Replies:** 4\ **Last updated:** [March 8, 2024, 1:36am UTC](https://dev-discuss.pytorch.org/t/torchinductor-update-7-key-optimizations-with-cpu-backend-in-pytorch-2-2-release/1867 "2024-03-08T01:36:17Z")
In our previous blogs of TorchInductor Update 4, TorchInductor Update 5 and TorchInductor Update 6, we detailed the progress and provided a technical deep dive into the optimization efforts for the Inductor C++/OpenMP ba… --- ## [Custom cuda extension support in Inductor](https://dev-discuss.pytorch.org/t/custom-cuda-extension-support-in-inductor/1924)
**Author:** [@youkaichao](https://dev-discuss.pytorch.org/u/youkaichao)\ **Replies:** 8\ **Last updated:** [March 7, 2024, 3:52pm UTC](https://dev-discuss.pytorch.org/t/custom-cuda-extension-support-in-inductor/1924 "2024-03-07T15:52:06Z")
Per this thread Inductor Triton Custom Op , inductor supports custom kernel written in triton. How about general cuda extension / kernel then? Is it possible to integrate them into inductor? How can inductor understand c… --- ## [Trying to understand flow for compilation](https://dev-discuss.pytorch.org/t/trying-to-understand-flow-for-compilation/1923)
**Author:** [@vims](https://dev-discuss.pytorch.org/u/vims)\ **Replies:** 1\ **Last updated:** [March 7, 2024, 11:42am UTC](https://dev-discuss.pytorch.org/t/trying-to-understand-flow-for-compilation/1923 "2024-03-07T11:42:34Z")
Hi, I’d like to understand how Inductor generates Triton kernels, starting from a function compiled with torch.compile all the way to a Triton kernel mapping. However, I’m kind of lost trying to find entry points. Even i… --- ## [Is there a plan for FX in a C++ IR?](https://dev-discuss.pytorch.org/t/is-there-a-plan-for-fx-in-a-c-ir/1889)
**Author:** [@Sujoy\_Saraswati](https://dev-discuss.pytorch.org/u/Sujoy_Saraswati)\ **Replies:** 2\ **Last updated:** [February 28, 2024, 3:39pm UTC](https://dev-discuss.pytorch.org/t/is-there-a-plan-for-fx-in-a-c-ir/1889 "2024-02-28T15:39:21Z")
Hi, Is there a way to consume the FX graphs directly from C++? As per How to get the computation graph and pass it to C++? - #2 by James\_Reed - FX (Functional Transformations) - PyTorch Forums, there was no plan for thi… --- ## [PyTorch/XLA 2.2 Release Dev Update](https://dev-discuss.pytorch.org/t/pytorch-xla-2-2-release-dev-update/1897)
**Author:** [@JackCaoG](https://dev-discuss.pytorch.org/u/JackCaoG)\ **Replies:** 0\ **Last updated:** [February 23, 2024, 12:30am UTC](https://dev-discuss.pytorch.org/t/pytorch-xla-2-2-release-dev-update/1897 "2024-02-23T00:30:26Z")
Hey I am here to give another dev update for the PyTorch/XLA 2.2 release. It is a relatively short update compared to the 2.1 because it only covers 3 months of development and it overlaps with a bunch of vacations at th… --- ## [Example inputs to compilers are now fake tensors](https://dev-discuss.pytorch.org/t/example-inputs-to-compilers-are-now-fake-tensors/990)
**Author:** [@ezyang](https://dev-discuss.pytorch.org/u/ezyang)\ **Replies:** 9\ **Last updated:** [February 13, 2024, 12:15am UTC](https://dev-discuss.pytorch.org/t/example-inputs-to-compilers-are-now-fake-tensors/990 "2024-02-13T00:15:40Z")
(Editor’s note: I meant to send this in December, but forgot. Here you go, later than it should have been!) The merged PR at Use dynamo fake tensor mode in aot\_autograd, move aot\_autograd compilation to lowering time \[M… --- ## [TorchDynamo Update 4: LazyTensor & nvFuser Experiments](https://dev-discuss.pytorch.org/t/torchdynamo-update-4-lazytensor-nvfuser-experiments/496)
**Author:** [@jansel](https://dev-discuss.pytorch.org/u/jansel)\ **Replies:** 4\ **Last updated:** [February 9, 2024, 5:36am UTC](https://dev-discuss.pytorch.org/t/torchdynamo-update-4-lazytensor-nvfuser-experiments/496 "2024-02-09T05:36:15Z")
Recap Since September 2021, we have working on an experimental project called TorchDynamo. TorchDynamo is a Python-level JIT compiler designed to make unmodified PyTorch programs faster. TorchDynamo hooks into the fra… --- ## [Cannot mutate inputs in aot\_export workflow](https://dev-discuss.pytorch.org/t/cannot-mutate-inputs-in-aot-export-workflow/1865)
**Author:** [@anieto](https://dev-discuss.pytorch.org/u/anieto)\ **Replies:** 0\ **Last updated:** [February 7, 2024, 1:45am UTC](https://dev-discuss.pytorch.org/t/cannot-mutate-inputs-in-aot-export-workflow/1865 "2024-02-07T01:45:37Z")
What’s the rationale for disabling mutations to user inputs while tracing via torch.export? Is it a soundness issue? Not implemented? Something else? Context: I’m trying to run a modified version of Llama through torch… --- ## [Developer docs for PyTorch inductor?](https://dev-discuss.pytorch.org/t/developer-docs-for-pytorch-inductor/1714)
**Author:** [@anubane](https://dev-discuss.pytorch.org/u/anubane)\ **Replies:** 3\ **Last updated:** [February 6, 2024, 6:58am UTC](https://dev-discuss.pytorch.org/t/developer-docs-for-pytorch-inductor/1714 "2024-02-06T06:58:38Z")
Are there any developer docs or guidelines regarding how a custom backend may be developed for inductor. In general is there a way to understand the code structure at Inductor CodeBase? I understand that there are some… --- ## [Compiling the optimizer with PT2](https://dev-discuss.pytorch.org/t/compiling-the-optimizer-with-pt2/1669)
**Author:** [@mlazos](https://dev-discuss.pytorch.org/u/mlazos)\ **Replies:** 8\ **Last updated:** [January 29, 2024, 10:24pm UTC](https://dev-discuss.pytorch.org/t/compiling-the-optimizer-with-pt2/1669 "2024-01-29T22:24:06Z")
TLDR; Compiling optimizers improved performance on all benchmarks: HuggingFace +18%, TorchBench +19%, and TIMM +8% E2E Compiled optimizers now support cudagraphs Averaged over all models in a test suite, average compil… --- ## [FYI: Many Dynamo tests were erroneously passing](https://dev-discuss.pytorch.org/t/fyi-many-dynamo-tests-were-erroneously-passing/1849)
**Author:** [@zou3519](https://dev-discuss.pytorch.org/u/zou3519)\ **Replies:** 0\ **Last updated:** [January 29, 2024, 9:26pm UTC](https://dev-discuss.pytorch.org/t/fyi-many-dynamo-tests-were-erroneously-passing/1849 "2024-01-29T21:26:46Z")
FYI: Many Dynamo tests were erroneously passing with: @voz PyTorch CI runs regular PyTorch unittests with Dynamo enabled (i.e. PYTORCH\_TEST\_WITH\_DYNAMO=1). Previously: these ran with suppress\_errors=True, which ignore… --- ## [\`compile\_autograd\`](https://dev-discuss.pytorch.org/t/compile-autograd/1840)
**Author:** [@jeromeku](https://dev-discuss.pytorch.org/u/jeromeku)\ **Replies:** 0\ **Last updated:** [January 26, 2024, 7:20pm UTC](https://dev-discuss.pytorch.org/t/compile-autograd/1840 "2024-01-26T19:20:47Z")
What is the purpose of the compile\_autograd argument in the dynamo benchmarking\` script? I thought that forward / backwards graphs were automatically compiled for a nn.Module when using torch.compile (with default induc… --- ## [Micro-optimizations for the most micro of benchmarks](https://dev-discuss.pytorch.org/t/micro-optimizations-for-the-most-micro-of-benchmarks/1836)
**Author:** [@jansel](https://dev-discuss.pytorch.org/u/jansel)\ **Replies:** 0\ **Last updated:** [January 25, 2024, 5:10am UTC](https://dev-discuss.pytorch.org/t/micro-optimizations-for-the-most-micro-of-benchmarks/1836 "2024-01-25T05:10:40Z")
Most of the benchmarking in PyTorch 2 has focused on large models taken from real-world applications. For this note, I want to take the completely opposite approach and instead focus on fixed overheads. Basically everyth… --- ## [\`torch.compile\` \`AOTAutograd\` backwards \_inductor function](https://dev-discuss.pytorch.org/t/torch-compile-aotautograd-backwards-inductor-function/1831)
**Author:** [@jeromeku](https://dev-discuss.pytorch.org/u/jeromeku)\ **Replies:** 0\ **Last updated:** [January 23, 2024, 11:05pm UTC](https://dev-discuss.pytorch.org/t/torch-compile-aotautograd-backwards-inductor-function/1831 "2024-01-23T23:05:07Z")
How can I get a runnable copy of the backwards graph when running torch.compile? For example: @torch.compile def f(x): out = x.sin() + x.cos() return out x = torch.ones(2, requires\_grad=True).cuda() out = f(x)… --- ## [\[GUIDE\] Getting C++ custom ops to work with torch.compile](https://dev-discuss.pytorch.org/t/guide-getting-c-custom-ops-to-work-with-torch-compile/1737)
**Author:** [@zou3519](https://dev-discuss.pytorch.org/u/zou3519)\ **Replies:** 3\ **Last updated:** [January 16, 2024, 7:22pm UTC](https://dev-discuss.pytorch.org/t/guide-getting-c-custom-ops-to-work-with-torch-compile/1737 "2024-01-16T19:22:37Z")
If you have a library of existing C++ custom ops that work with PyTorch, they may not work with torch.compile. Here’s our guide on how to get them to work: The C++ Custom Operators Manual - Google Docs . We’re still loo… --- ## [\[RFC\] LazyScheduler for operator reordering](https://dev-discuss.pytorch.org/t/rfc-lazyscheduler-for-operator-reordering/1808)
**Author:** [@yf225](https://dev-discuss.pytorch.org/u/yf225)\ **Replies:** 0\ **Last updated:** [January 13, 2024, 6:09am UTC](https://dev-discuss.pytorch.org/t/rfc-lazyscheduler-for-operator-reordering/1808 "2024-01-13T06:09:50Z")
Motivation Under both eager mode and torch.compile, there are use cases to reorder operators (in compile case, across graphs) without model code changes, and existing APIs don’t serve them well. For OSS models: Some u… --- ## [Decomposition slows down the lazy tensor tracing](https://dev-discuss.pytorch.org/t/decomposition-slows-down-the-lazy-tensor-tracing/1788)
**Author:** [@AngWang](https://dev-discuss.pytorch.org/u/AngWang)\ **Replies:** 5\ **Last updated:** [January 11, 2024, 7:30am UTC](https://dev-discuss.pytorch.org/t/decomposition-slows-down-the-lazy-tensor-tracing/1788 "2024-01-11T07:30:08Z")
The decomposition slows down the lazy tensor tracing when running with TorchXLA. Here is the timeline with stack info: Removing the decomposition-related code resolves the issue: This is the timeline after removi… [Previous page](https://dev-discuss.pytorch.org/c/compiler/5.md?page=2) [Next page](https://dev-discuss.pytorch.org/c/compiler/5.md?page=4)