# PrimTorch: How backend/compiler writers interact with various IRs **URL:** \ **Category:** compiler\ **Created:** [March 8, 2023, 3:56am UTC](https://dev-discuss.pytorch.org/t/primtorch-how-backend-compiler-writers-interact-with-various-irs/1111 "2023-03-08T03:56:59Z")\ **Posts on this page:** 1\ **Showing post:** 2
**Author:** ![gilfree](https://yyz2.discourse-cdn.com/flex036/user_avatar/dev-discuss.pytorch.org/gilfree/32/786_2.png) [@gilfree](https://dev-discuss.pytorch.org/u/gilfree)\ **Post date:** [March 12, 2023, 2:57pm UTC](https://dev-discuss.pytorch.org/t/primtorch-how-backend-compiler-writers-interact-with-various-irs/1111/2 "2023-03-12T14:57:54Z")
> [@Minerva\_Yu](#): > > For Core ATen IR is the subset of aten operators, the way I use it is the same with former, i.e. through “native\_functions.yaml”. ( But how could I distinguish who are the “Core ATen” ops and who are the “ATen” ones? 🧐) Hi @Minerva_Yu, No answers, just hints - I am at the same situation as you are - trying to understand how should I integrate my compiler with the torch IR. I got some pointers here: > > > \### 🐛 Describe the bug > > When using \`TorchRefsMode\`, aot\_function cannot handle \`…reshape\`. > > I am not sure that my usage is correct, but my goal is to decompose a graph to prim ops, in order to later export outside pytorch. > > I tried the code below, and an internal assert is triggered, with request to open a bug: > > \`\`\`console > RuntimeError: !schema.hasAnyAliasInfo() INTERNAL ASSERT FAILED at "../aten/src/ATen/FunctionalizeFallbackKernel.cpp":33, please report a bug to PyTorch. mutating and aliasing ops should all have codegen'd kernels > \`\`\` > > When not using fake tensor mode I get some error in a check for \`is\_quantized\` at some place. > > If there is a "correct" way to get a prim graph, I would very much like to know what it is. > > Code: > \`\`\`python > from typing import List > import torch > from torch import Tensor > from functorch.compile import aot\_function > from torch.\_subclasses.fake\_tensor import FakeTensorMode > from torch.\_prims.context import TorchRefsMode > > def func(scale: Tensor,shape): > return torch.reshape(scale,shape) > > > def custom\_backend(gm: torch.fx.GraphModule, example\_inputs: List\[torch.Tensor\]): > print("custom backend called with FX graph:") > gm.graph.print\_tabular() > return gm.forward > > scale=torch.ones(3) > > > with FakeTensorMode(allow\_non\_fake\_inputs=True): > scale=scale.clone() > > with TorchRefsMode(strict=True): > func = aot\_function(func,fw\_compiler=custom\_backend) > print(func(scale,shape=(-1,))) > \`\`\` > > Stack trace below: > \`\`\`console > Traceback (most recent call last): > File "bug.py", line 39, in \ > print(func(scale,shape=(-1,))) > File ".../torch/\_functorch/aot\_autograd.py", line 2643, in returned\_function > compiled\_fn = create\_aot\_dispatcher\_function( > File ".../torch/\_dynamo/utils.py", line 163, in time\_wrapper > r = func(\*args, \*\*kwargs) > File ".../torch/\_functorch/aot\_autograd.py", line 2491, in create\_aot\_dispatcher\_function > compiled\_fn = compiler\_fn(flat\_fn, fake\_flat\_args, aot\_config) > File ".../torch/\_functorch/aot\_autograd.py", line 1802, in aot\_wrapper\_dedupe > compiled\_fn = compiler\_fn(wrapped\_flat\_fn, deduped\_flat\_args, aot\_config) > File ".../torch/\_functorch/aot\_autograd.py", line 1278, in aot\_dispatch\_base > \_fw\_metadata, \_out = run\_functionalized\_fw\_and\_collect\_metadata( > File ".../torch/\_functorch/aot\_autograd.py", line 606, in inner > flat\_f\_outs = f(\*flat\_f\_args) > File ".../torch/\_functorch/aot\_autograd.py", line 1800, in wrapped\_flat\_fn > return flat\_fn(\*add\_dupe\_args(args)) > File ".../torch/\_functorch/aot\_autograd.py", line 2623, in flat\_fn > tree\_out = fn(\*args, \*\*kwargs) > File "/homes/giladf/dlo/dlo2/docs/development/design/bug.py", line 23, in func > return torch.reshape(scale,shape) > File ".../torch/\_prims/context.py", line 191, in \_\_torch\_function\_\_ > return func(\*args, \*\*kwargs) > File ".../torch/\_refs/\_\_init\_\_.py", line 3260, in reshape > return \_reshape\_view\_helper(a, \*shape, allow\_copy=True) > File ".../torch/\_refs/\_\_init\_\_.py", line 3156, in \_reshape\_view\_helper > return prims.view\_of(a) > File ".../torch/\_ops.py", line 284, in \_\_call\_\_ > return self.\_op(\*args, \*\*kwargs or {}) > File ".../torch/\_prims/context.py", line 172, in \_\_torch\_function\_\_ > return orig\_func(\*args, \*\*kwargs) > File ".../torch/\_ops.py", line 284, in \_\_call\_\_ > return self.\_op(\*args, \*\*kwargs or {}) > File ".../torch/\_prims/\_\_init\_\_.py", line 286, in \_autograd\_impl > return backwards\_not\_supported(\_prim)(\*args, \*\*kwargs) > File ".../torch/\_prims\_common/wrappers.py", line 320, in \_autograd\_impl > return redispatch\_prim(args, kwargs) > File ".../torch/\_prims\_common/wrappers.py", line 290, in redispatch\_prim > return prim(\*args, \*\*kwargs) > File ".../torch/\_ops.py", line 284, in \_\_call\_\_ > return self.\_op(\*args, \*\*kwargs or {}) > RuntimeError: !schema.hasAnyAliasInfo() INTERNAL ASSERT FAILED at "../aten/src/ATen/FunctionalizeFallbackKernel.cpp":33, please report a bug to PyTorch. mutating and aliasing ops should all have codegen'd kernels > \`\`\` > > \### Versions > > Collecting environment information... > PyTorch version: 2.0.0+cu117 > Is debug build: False > CUDA used to build PyTorch: 11.7 > ROCM used to build PyTorch: N/A > > OS: Ubuntu 18.04.6 LTS (x86\_64) > GCC version: (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0 > Clang version: 9.0.0 (tags/RELEASE\_900/final) > CMake version: version 3.25.0 > Libc version: glibc-2.27 > > Python version: 3.10.6 (main, Aug 30 2022, 16:00:07) \[GCC 7.5.0\] (64-bit runtime) > Python platform: Linux-5.15.87-051587-generic-x86\_64-with-glibc2.27 > Is CUDA available: True > CUDA runtime version: 11.7.64 > CUDA\_MODULE\_LOADING set to: LAZY > GPU models and configuration: > GPU 0: NVIDIA A100-PCIE-40GB > GPU 1: NVIDIA A100-PCIE-40GB > > Nvidia driver version: 525.78.01 > cuDNN version: Could not collect > HIP runtime version: N/A > MIOpen runtime version: N/A > Is XNNPACK available: True > > CPU: > Architecture: x86\_64 > CPU op-mode(s): 32-bit, 64-bit > Address sizes: 43 bits physical, 48 bits virtual > Byte Order: Little Endian > CPU(s): 64 > On-line CPU(s) list: 0-63 > Vendor ID: AuthenticAMD > Model name: AMD EPYC 7502 32-Core Processor > CPU family: 23 > Model: 49 > Thread(s) per core: 2 > Core(s) per socket: 32 > Socket(s): 1 > Stepping: 0 > BogoMIPS: 5000.28 > Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr\_opt pdpe1gb rdtscp lm constant\_tsc rep\_good nopl nonstop\_tsc cpuid extd\_apicid aperfmperf rapl pni pclmulqdq monitor ssse3 fma cx16 sse4\_1 sse4\_2 x2apic movbe popcnt aes xsave avx f16c rdrand lahf\_lm cmp\_legacy svm extapic cr8\_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt tce topoext perfctr\_core perfctr\_nb bpext perfctr\_llc mwaitx cpb cat\_l3 cdp\_l3 hw\_pstate ssbd mba ibrs ibpb stibp vmmcall fsgsbase bmi1 avx2 smep bmi2 cqm rdt\_a rdseed adx smap clflushopt clwb sha\_ni xsaveopt xsavec xgetbv1 xsaves cqm\_llc cqm\_occup\_llc cqm\_mbm\_total cqm\_mbm\_local clzero irperf xsaveerptr rdpru wbnoinvd arat npt lbrv svm\_lock nrip\_save tsc\_scale vmcb\_clean flushbyasid decodeassists pausefilter pfthreshold avic v\_vmsave\_vmload vgif v\_spec\_ctrl umip rdpid overflow\_recov succor smca sme sev sev\_es > Virtualization: AMD-V > L1d cache: 1 MiB (32 instances) > L1i cache: 1 MiB (32 instances) > L2 cache: 16 MiB (32 instances) > L3 cache: 128 MiB (8 instances) > NUMA node(s): 1 > NUMA node0 CPU(s): 0-63 > Vulnerability Itlb multihit: Not affected > Vulnerability L1tf: Not affected > Vulnerability Mds: Not affected > Vulnerability Meltdown: Not affected > Vulnerability Mmio stale data: Not affected > Vulnerability Retbleed: Mitigation; untrained return thunk; SMT enabled with STIBP protection > Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl and seccomp > Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and \_\_user pointer sanitization > Vulnerability Spectre v2: Mitigation; Retpolines, IBPB conditional, STIBP always-on, RSB filling, PBRSB-eIBRS Not affected > Vulnerability Srbds: Not affected > Vulnerability Tsx async abort: Not affected > > Versions of relevant libraries: > \[pip3\] mypy==1.0.1 > \[pip3\] mypy-extensions==1.0.0 > \[pip3\] numpy==1.24.1 > \[pip3\] pytorch-lightning==1.9.3 > \[pip3\] pytorch-triton==2.0.0+d54c04abe2 > \[pip3\] torch==2.0.0+cu117 > \[pip3\] torchaudio==2.0.0+cu117 > \[pip3\] torchmetrics==0.11.1 > \[pip3\] torchvision==0.15.0+cu117 > \[conda\] Could not collect > > cc @ezyang @mruberry @ngimel @Lezcano @peterbell10 You can play with the backend and get only torch.xxx with no aten, or only aten if you run the aot compiler. It seems that to get the core aten and prims - you can use what @ezyang pointed there - using proxy\_tensor, make\_fx & TorchRefsMode, but I am not sure how these levels relate to [IRs — PyTorch master documentation](https://pytorch.org/docs/master/ir.html). Also see the post here: [PrimTorch: could we get pure core-aten-ops or prims-ops after aot\_autograd - #5 by SherlockNoMad](https://dev-discuss.pytorch.org/t/primtorch-could-we-get-pure-core-aten-ops-or-prims-ops-after-aot-autograd/1048/5), on how to get the core aten ir. Seems also that you can define the decompositions you want, in python, and get varying levels of IRs from torch.xxx to “core atens and prims” - See the notebook I linked to in that issue. --- _[View the full topic](https://dev-discuss.pytorch.org/t/primtorch-how-backend-compiler-writers-interact-with-various-irs/1111)._