# State of symbolic shapes branch **URL:** \ **Category:** compiler\ **Created:** [September 19, 2022, 5:32pm UTC](https://dev-discuss.pytorch.org/t/state-of-symbolic-shapes-branch/777 "2022-09-19T17:32:53Z")\ **Posts on this page:** 1\ **Showing post:** 9
**Author:** ![ezyang](https://yyz2.discourse-cdn.com/flex036/user_avatar/dev-discuss.pytorch.org/ezyang/32/12_2.png) [@ezyang](https://dev-discuss.pytorch.org/u/ezyang)\ **Post date:** [October 30, 2022, 7:30pm UTC](https://dev-discuss.pytorch.org/t/state-of-symbolic-shapes-branch/777/9 "2022-10-30T19:30:30Z")
# State of symbolic shapes branch: Oct 30 edition The symbolic-shapes branch (PyTorch: [Symbolic shapes by ezyang · Pull Request #84246 · pytorch/pytorch · GitHub](https://github.com/pytorch/pytorch/pull/84246)) is a long running branch containing a large number of features and bugfixes related to dynamic shapes support in PyTorch. Previous update: [State of symbolic shapes branch - #8 by ezyang](https://dev-discuss.pytorch.org/t/state-of-symbolic-shapes-branch/777/8) Commit ID at time of writing: 121e8ebcc2fc50e5ca28cfb3ad437596084424e1 ## Executive summary Voz enabled propagation of symbolic shapes in dynamo (previously, symbolic shapes were only propagated in AOTAutograd), and this uncovered a large number of previously undiscovered bugs and coverage problems in TORCHDYNAMO\_DYNAMIC\_SHAPES=1 itself. The team plans to pivot to working on the torchdynamo codebase to help fix these problems. - @SherlockNoMad found in [Meta OpInfo Test for stride correctness by SherlockNoMad · Pull Request #87849 · pytorch/pytorch · GitHub](https://github.com/pytorch/pytorch/pull/87849) that there are ~70 aten ops have mismatched stride value between meta function and eager’s implementation. Usually, this is a result of incomplete python meta function, or decompositions (because our unit tests was not asserting on stride’s correctness, and we didn’t have enough test cases for strided inputs). Sherlock compiled all the failures into this tracker sheet [Stride Mismatch Tracker - Google Sheets](https://docs.google.com/spreadsheets/d/1Qa6ACLeHtVCLyiHQWZCie76QaRPlGwk1-5USI3ktM7U/edit) . Bugs on this sheet are open season for burn down. - **Model training status on symbolic-shapes.** (@ezyang) See also [Operators that need symbolic support - Google Sheets](https://docs.google.com/spreadsheets/d/1-ghURoD6tzTushbCb22tFd5B5uNg7HSxVG4dQBwoaFA/edit?pli=1#gid=401920381) (out of date). - aot\_eager, no TDS: 149 out of 165 (new) - [logs](https://gist.github.com/ezyang/fd6cfbf4cd8956f1a6e0933aa86c2de6) - aot\_eager, with TDS: 118 out of 163 (+2 WoW; heavily depressed due to new bugs discovered in torchdynamo) - [logs](https://gist.github.com/ezyang/2c6e87f66fee1abf6c1b0a536bb23d8c) - inductor, with TDS: 45 out of 163 (new) - [logs](https://gist.github.com/ezyang/1be7475e7303fb9190bcb7729cbba257) - **Model inference status on symbolic-shapes.** (@ezyang) - inductor, with TDS: 69 out of 177 (new) - [logs](https://gist.github.com/b7f473fd34d9bd96a935c538d80885ad) - **Model inference status on master.** (@ezyang) - This week is really bad, as we haven’t gotten all the branch fixes after dynamo symbolic shapes fallout. With TORCHDYNAMO\_DYNAMIC\_SHAPES=1: 4 out of 177 (-23 WoW) - [logs](https://gist.github.com/ezyang/bd544312dbb5280fb307dd48d72d9ae8) - **OpInfo tests on symbolic-shapes.** (@ezyang) - `pytest -v test/test_proxy_tensor.py -k test_make_fx_symbolic_exhaustive` - 347 passed (+8 WoW), 374 failed (-7 WoW), 497 skipped (+1 WoW) - `pytest -v test/functorch/test_aotdispatch.py -k test_aot_autograd_symbolic_exhaustive` - 238 passed (+19 WoW), 246 failed (-18 WoW), 125 skipped (+0 WoW) Previous branch diff: 70 files changed, 1681 insertions(+), 430 deletions(-) Current branch diff: 110 files changed, 2788 insertions(+), 2114 deletions(-) ## Notable bug fixes - Dynamo’s dynamic shapes support is seriously buggy, and it doesn’t seem like there is a clear enough conceptual framework for how things should be implemented to make bug fixes simple. For example, voz in [https://github.com/pytorch/pytorch/pull/84246/commits/7b72a45eb1f716e8b8d5f74b1f8f887de617674a](https://github.com/pytorch/pytorch/pull/84246/commits/7b72a45eb1f716e8b8d5f74b1f8f887de617674a) needed to write a page of code just to get dynamic size(i) method calls working. We intend to have a knowledge sharing session with Voz on Tuesday to get the team up to speed on how to approach dynamo bugs. - We’re still fixing incorrect stride bugs. [Delete incorrect and duplicate meta\_add\_](https://github.com/pytorch/pytorch/pull/84246/commits/8ed124382aae70f3918c5e654d626b01b554469a) in particular was a symbolic-shapes branch only howler. It is still quite difficult to diagnose these without a minifier; ezyang proposed we have a mode where aot\_eager checks the real tensors have metadata consistent with the trace, but this still isn’t implemented yet. Sherlock has a workstream for fixing strides based on the test suite: [Stride Mismatch Tracker - Google Sheets](https://docs.google.com/spreadsheets/d/1Qa6ACLeHtVCLyiHQWZCie76QaRPlGwk1-5USI3ktM7U/edit?usp=sharing) There is still a live problem with softmax: minimal repro [gist:54f03e02fd36069bf9693ae2ab707d10 · GitHub](https://gist.github.com/ezyang/54f03e02fd36069bf9693ae2ab707d10) - In parallel to Sherlock’s burn down of stride bugs, @ezyang is investigating whether or not we can make incorrect stride bugs less severe by preserving reshapes during tracing. This is tricky to do, because we still have to do autograd, and autograd doesn’t directly support reshape (as it sometimes returns a view and sometimes returns a fresh tensor). Our plan is to transform reshape into an always copying operation, and adding enough extra sanity checking in functionalization to detect if this is not semantics preserving. To do this, we need to run functionalization and autograd at the same time; thus [functionalize and compute joint simultaneously](https://github.com/pytorch/pytorch/pull/84246/commits/28168663c75faa60e45c93951126d749a955dcae). Implementing this was a doozy, as it triggered multiple functionalization bugs: - FunctionalTensorWrapper that directly wraps fake tensor didn’t report their devices correctly, and so went to the wrong kernels. After staring at the backtrace, this was determined by code reading. We had to both [forward device calls to inner tensor in FunctionalTensorWrapper](https://github.com/pytorch/pytorch/pull/84246/commits/4d4e2a5e6ce0ac72c8aa3709e4e6bb582f1fb602) and also [update functionalization dispatch key strategy](https://github.com/pytorch/pytorch/pull/84246/commits/0bb517d33b0f854499d47438bacdebf8367b3f08) so that FunctionalTensorWrapper appropriately pretends to be a Dense tensor. - We made a LOT of quality of life improvements to the branch this week. A lot of it was simply dogfooding the software and making adjustments when we noticed things could be improved. These range from paper cuts (spammy warnings, overly verbose exceptions) to important debugging tools like (printing more program state on failure, e.g., the incomplete make\_fx traced graph or strides of variables in a graph) to important unblocking features (adding timeout so that sympy infinite loops don’t block model sweeps). - [Use functionalize in symbolic exhaustive tests, not sound otherwise](https://github.com/pytorch/pytorch/pull/84246/commits/d0b2f80479b55540630fec5bc71988f9d4a5e16c) is a howler; someone intentionally turned off functionalization on the test suite, so many tests were failing because AOTAutograd without functionalization isn’t actually sound. Fixing this helped a number of test cases pass. The moral of the story is, don’t give users configuration knobs that are known unsound, unless you make it really obvious (or at least, obvious enough that a core dev doesn’t turn that knob on by default in tests, and another core dev approves it in CR.) - [Fix bernoulli functionalization](https://github.com/pytorch/pytorch/pull/84246/commits/bb6e8dfb39e903b853fe7ff3683da9d8c9c701bb) was an annoying typofix in the bernoulli implementation for a longstanding problem (long enough that XLA had manually worked around it.) It had to be diagnosed manually (by looking at the graphs and noticing some operators were missing from DCE); it could have been immediately caught by testing for no mutating ops after functionalization, but the PR that implemented this is still not landed [aot\_autograd: add assert for functional-only graph by bdhirsh · Pull Request #85681 · pytorch/pytorch · GitHub](https://github.com/pytorch/pytorch/pull/85681) (I must emphasize how important it is to actually land the PRs you write!) - A runtime error complaining about shape mismatch in backwards turned out to be a functionalization bug, where we were not copying enough metadata when doing a shallow copy and detach of FunctionalTensorWrapper (which happens when a variable is saved for backwards.) Both bdhirsh and ezyang root caused the problem at about the same time. Fixed in [functionalization: fix detach()](https://github.com/pytorch/pytorch/pull/84246/commits/74d57e1cbfb6dfad512bb6c622ce7b6e54610d88) and tested by [Saved tensor that is view test case functionalization](https://github.com/pytorch/pytorch/pull/84246/commits/b6d8182158db0f1b43ffbed74571f7c5ac65ca95) - This is not a bug fix per se, but an infrastrucutre improvement [basic bin\_op(symint, symint) support](https://github.com/pytorch/pytorch/pull/84246/commits/69b465df59f1ad89fd291d02bf20bc33c2a9b621) was initially done in a brute force way by adding each overload for mixed operations one-by-one, but on subsequent code review we didn’t like it. [Unify SymIntNode and SymFloatNode into SymNode](https://github.com/pytorch/pytorch/pull/84246/commits/45ec93c4e1f6743ef76aec600b6982b06593a0f2) instead removed the static types from C++, which meant you don’t have to add overloads. This produced a patch that was functionally equivalent, but a net decrease in total LOC. [https://twitter.com/ezyang/status/1585373693852934144](https://twitter.com/ezyang/status/1585373693852934144) - [Prevent use of as\_strided in backwards](https://github.com/pytorch/pytorch/pull/84246/commits/df5637ac067484168bb7a153ad8b531c90d6436a) was discovered when inspecting some backward graphs by hand and noticing they had a lot of `as_strided` calls in them. For reference, typical user programs don’t really ever use `as_strided`, and neither do most of our autograd derivative formulas, so it is weird that they were showing up. In fact, this is due to how our view mutation logic work, which by default converts all views into AsStrided operations so you don’t have to track the provenance of any single view. This is a nice performance optimization for eager, but it generates code that’s worse for compilers, and in fact, XLA already had a way of disabling this shortcut and maintaining the views by hand. Turning this on for PT2 eliminates these as strided calls. BTW, as strided calls are preferably avoided in IR as they are extremely input stride sensitive; if the incoming tensor has a different stride, as strided will silently corrupt your data. This would be less of a problem with a “compositional” variant of as strided that respects input strides (e.g., if a dim was already stride 2, restriding it by 2 would result in 2 \* 2). - We have a lot of sympy infinite loops. [Fix another sympy infinite loop](https://github.com/pytorch/pytorch/pull/84246/commits/c0b9f783748657b7c895a6ce909993afa506cb8e) whackamoles one of them, but there are still more. @Chillee we need to figure out a more sustainable strategy for these. - [Disable aot autograd cache](https://github.com/pytorch/pytorch/pull/84246/commits/7f82efeb1cfbc3a9707e1e8af3ff4c79c6673f38) is a nice stopgap: some of our models are now failing because they _do_ exercise dynamic shapes, but our guards are insufficient (because AOTAutograd guards aren’t propagated up to torchdynamo yet.) Disabling the AOTAutograd cache means we always recompile at AOTAutograd until this can be fixed. - [Unify meta tensor and fake tensor converter conversion](https://github.com/pytorch/pytorch/pull/84246/commits/22b9ee8ab76811f30509d14af857d3073c2fce62) broke a number of inductor model runs on master (non-dynamic shapes configuration.) I was able to fix this by disabling meta tensor converter’s view preservation (it’s logic to remake a tensor into a view if it was originally a view). But in principle, it should be OK to do this. This is very perplexing. Some of the current investigation notes are at [GitHub · Where software is built](https://github.com/pytorch/torchdynamo/issues/1815) ## What’s new on the branch this week? This time, instead of doing commits chronologically, I’ve tried grouping them by theme. Meta support - [Fix meta for meta\_fill\_](https://github.com/pytorch/pytorch/pull/84246/commits/aa865e2e32de06f4905319de698e061c8512bad9) SherlockNoMad - [Delete incorrect and duplicate meta\_add\_](https://github.com/pytorch/pytorch/pull/84246/commits/8ed124382aae70f3918c5e654d626b01b554469a) ezyang/Chillee - [meta for topk](https://github.com/pytorch/pytorch/pull/84246/commits/4c3abbe49ff22dc10c4493f52df030f5fd7b48c7) ezyang - [Logical binary inplace](https://github.com/pytorch/pytorch/pull/84246/commits/27d6c50937f73ea13a70e38ed39577a8886329dc) ezyang - [Implement scalar\_tensor meta](https://github.com/pytorch/pytorch/pull/84246/commits/606b499f502f48f1e1044b1254a1591344574a72) ezyang - [fix meta defaults in meta\_\_fused\_moving\_avg\_obs\_fq\_helper](https://github.com/pytorch/pytorch/pull/84246/commits/4e9225d53949e54eb47eca682740eb9911dfe85f) bdhirsh SymInt support - [Symintify tensor\_split](https://github.com/pytorch/pytorch/pull/84246/commits/32b837773ff24ed15d938403eda99a8168a9af54) ezyang - [SymInt padding/output padding on convolution; default arg codegen](https://github.com/pytorch/pytorch/pull/84246/commits/ebe8f76c31731fe1394bcc273ba5dde119c2fc74) ezyang - [adaptive\_avg\_pool2d symintification continued](https://github.com/pytorch/pytorch/pull/84246/commits/ed32bb3e135cdbc2de20b36cc6f206f79ca29b3d) anjali411 - [Some more meta support for unit tests test\_aotdispatch](https://github.com/pytorch/pytorch/pull/84246/commits/bccaa4f9b775b0e598acf1cb68f1ee1f11e5d09d) ezyang - [More progress on quantized](https://github.com/pytorch/pytorch/pull/84246/commits/23c9ef06c4d8f7216e33089f5c6857932f935f17) ezyang - [pass on aten.\_fused\_moving\_avg\_obs\_fq\_helper\_functional.default](https://github.com/pytorch/pytorch/pull/84246/commits/b696033fbabea20c06d49ec37a611ab466bff284) ezyang - [SymIntify resize\_](https://github.com/pytorch/pytorch/pull/84246/commits/19254d19aae2ba792fd22ff4c2fff1555c2df6d6) ezyang - [SymIntify \_copy functionalization kernels (and \_copy\_out too)](https://github.com/pytorch/pytorch/pull/84246/commits/f48ff123ddb156fb38a531026b6286bf587b54c4) ezyang - [Sym support for torch.numel](https://github.com/pytorch/pytorch/pull/84246/commits/84ad3a88918044cf57993bcd9eafe29768392a33) ezyang Quality of life - [[HACK] Stop rethrowing exceptions](https://github.com/pytorch/pytorch/pull/84246/commits/7c51fdabe2b9887da0b22fae39ea3a4b96a16617) ezyang - [[HACK] Make it easier to correlate what you ran and the status](https://github.com/pytorch/pytorch/pull/84246/commits/5548c70f852d72658f59522cc4a71d2e717a40cf) ezyang - [enable pybind debugging by default](https://github.com/pytorch/pytorch/pull/84246/commits/ee96627130b703906fd425e71da47541ae5df2ab) albanD - [[HACK] Don’t rethrow TorchRuntimeError](https://github.com/pytorch/pytorch/pull/84246/commits/fe29040c42a9cfd0ceadd7152320634630387194) ezyang (plus bug fix [Fix missing re-raise in exception fallthrough](https://github.com/pytorch/pytorch/pull/84246/commits/5494bfb654e196c3c2b0a1a1feb72db073789077) ) - [Improve broadcast\_shapes error printing](https://github.com/pytorch/pytorch/pull/84246/commits/8465f40db05ac8c124bbcd16d4cf707aaef6849f) ezyang - [[HACK] Print size hints, not expressions](https://github.com/pytorch/pytorch/pull/84246/commits/9eb36ef2266f1648af71e6b10fa3a00197238aa3) ezyang - [[HACK] Make logging less spammy](https://github.com/pytorch/pytorch/pull/84246/commits/47ce2afd236ec743b3901e849c4c09daea9f3602) ezyang - [Make FX collected traceback more useful by stripping format\_stack() f…](https://github.com/pytorch/pytorch/pull/84246/commits/0ea3e457721c922cb47ea899e48122f3d6eae013) ezyang - [[HACK] Record stack traces on make\_fx by default.](https://github.com/pytorch/pytorch/pull/84246/commits/78ec08b4b62c7d99828b9e70d5800a66178d767d) ezyang - [[HACK] Print incomplete graph upon failure](https://github.com/pytorch/pytorch/pull/84246/commits/bc6f6f3b44f3f9ac21129e60e0976fe1754e02c2) ezyang - [Added stride printing to gm.print\_readable() and removed add\_meta met…](https://github.com/pytorch/pytorch/pull/84246/commits/dfc9a3f3d50d5710a386e8ca8fcb363461811dbc) Chillee + [update print\_readable](https://github.com/pytorch/pytorch/pull/84246/commits/7e8fa8dda6092e276ee29d9be265a0fcb4565b27) anjali411 - [Ignore log files](https://github.com/pytorch/pytorch/pull/84246/commits/60559b00b3781deca37494a65fbbd9c9ad7cd5a4) ezyang - [Add a 5min timeout](https://github.com/pytorch/pytorch/commit/3147e631eee1f23723c81635a3ddc90cda8428b6) ezyang - [Distinguish timeout and regular fail, go to stderr](https://github.com/pytorch/pytorch/pull/84246/commits/827d86a9127f8f10e176a7981dcdcd159157f083) ezyang - [[HACK] Disable High loss value assert](https://github.com/pytorch/pytorch/pull/84246/commits/49c676f19073734078c347a035dfc7838b3e298c) ezyang - [Report what node failed when running an operation](https://github.com/pytorch/pytorch/pull/84246/commits/6037b1823ecab5852c68442d584c92e10a2680e5) ezyang - [Beefier error message](https://github.com/pytorch/pytorch/pull/84246/commits/e5702e57aad80c32c7abcd7816b56ba136aec28b) ezyang - [Be a little more accurate in the report](https://github.com/pytorch/pytorch/pull/84246/commits/2a870b14e15f0d5c670fc3480ad54c0b70fd78ac) ezyang Dynamo - [Partial cherry pick from vision\_maskrcnn bugfix](https://github.com/pytorch/pytorch/pull/84246/commits/666c4f5c87423880fe077f6992994df4e6c34fdc) ezyang - [[Dynamo] Symbolic shape guards (](https://github.com/pytorch/pytorch/pull/84246/commits/38ddcae129d73a368a55f84f4d065af56147c7d1) voz - Misc dynamo fixes [wip](https://github.com/pytorch/pytorch/pull/84246/commits/15894a6b0930d208a61c632568a94fb736defad1) [cleanups](https://github.com/pytorch/pytorch/pull/84246/commits/7981239e5c8c87333689a50a300710ee3990e82c) [rm comment](https://github.com/pytorch/pytorch/pull/84246/commits/754a0ab186eac67636fca42aa733faf803195cb5) voz - [[dynamo] Refactor DynamicShapeVariable to be a top level VariableTracker](https://github.com/pytorch/pytorch/pull/87810) voz (voz/fix\_vars branch merge) - [Remove some unnecessary BC code](https://github.com/pytorch/pytorch/pull/84246/commits/d361636ef0547bcff4e97d5f9802e7920b775162) ezyang - [Enable Python dispatcher when running fake tensors in dynamo](https://github.com/pytorch/pytorch/pull/84246/commits/463fb620aea4f90111b54d07cb35172336a7a40e) ezyang - [Fix floor guards in torchdynamo guard codegen](https://github.com/pytorch/pytorch/pull/84246/commits/bd34f4cfe7824d8476e217d9a2d562a7ad8b661e) ezyang - [Fix slice](https://github.com/pytorch/pytorch/pull/84246/commits/5448b5e5a21632827685dac1cb9995724407764e) voz - [Delete questionable symintification of constant](https://github.com/pytorch/pytorch/pull/84246/commits/eab3b9ad0f30452b541372455d04bd92ec1e6a93) ezyang - [Fix dynamo not recording slices in graphs. This led to issues with mi…](https://github.com/pytorch/pytorch/pull/84246/commits/297c2a89b19b6b466f3599d4c9c29cacebecc336) ( [Fix dynamo not recording slices in graphs](https://github.com/pytorch/pytorch/pull/87945)) voz - [Specialize on dynamic shape control flow instead of graph breaking](https://github.com/pytorch/pytorch/pull/88039) voz - [Fix math.sqrt and size(i) handling in dynamo (](https://github.com/pytorch/pytorch/pull/84246/commits/7b72a45eb1f716e8b8d5f74b1f8f887de617674a) [Fix](https://github.com/pytorch/pytorch/pull/84246/commits/121e8ebcc2fc50e5ca28cfb3ad437596084424e1) voz Functionalization - [Use functionalize in symbolic exhaustive tests, not sound otherwise](https://github.com/pytorch/pytorch/pull/84246/commits/d0b2f80479b55540630fec5bc71988f9d4a5e16c) ezyang - [Fix bernoulli functionalization.](https://github.com/pytorch/pytorch/pull/84246/commits/bb6e8dfb39e903b853fe7ff3683da9d8c9c701bb) ezyang - [Forward device calls to inner tensor in FunctionalTensorWrapper](https://github.com/pytorch/pytorch/pull/84246/commits/4d4e2a5e6ce0ac72c8aa3709e4e6bb582f1fb602) ezyang - [Update functionalization dispatch key strategy](https://github.com/pytorch/pytorch/pull/84246/commits/0bb517d33b0f854499d47438bacdebf8367b3f08) ezyang - [Functionalize and compute joint simultaneously](https://github.com/pytorch/pytorch/pull/84246/commits/28168663c75faa60e45c93951126d749a955dcae) ezyang - [Save and restore reapply views TLS correctly](https://github.com/pytorch/pytorch/pull/84246/commits/fbc9a682cbe38b1ade4bded286a1ccfef0034798) ezyang - [Saved tensor that is view test case functionalization](https://github.com/pytorch/pytorch/pull/84246/commits/b6d8182158db0f1b43ffbed74571f7c5ac65ca95) ezyang - [functionalization: fix detach()](https://github.com/pytorch/pytorch/pull/84246/commits/74d57e1cbfb6dfad512bb6c622ce7b6e54610d88) bdhirsh Infrastructure - [basic bin\_op(symint, symint) support](https://github.com/pytorch/pytorch/pull/84246/commits/69b465df59f1ad89fd291d02bf20bc33c2a9b621) [more basic bin\_op(symint, symint) support](https://github.com/pytorch/pytorch/pull/84246/commits/9504b5d93d1e0a65c32d3dbf9636e4effe394dea) bdhirsh, subsumed by [Unify SymIntNode and SymFloatNode into SymNode](https://github.com/pytorch/pytorch/pull/84246/commits/45ec93c4e1f6743ef76aec600b6982b06593a0f2) ezyang - [[HACK] Prevent use of as\_strided in backwards](https://github.com/pytorch/pytorch/pull/84246/commits/df5637ac067484168bb7a153ad8b531c90d6436a) ezyang - [Do not use unsafe restriding for subclasses](https://github.com/pytorch/pytorch/pull/84246/commits/11637f6e76d566a793d92640001f945ad0b8bf91) ezyang - [Add get\_guard\_expr to symbolic\_shapes which returns all guards in a s…](https://github.com/pytorch/pytorch/pull/84246/commits/3b0df231b1fea49cf6a07f57cfbddc53f7eab4fa) Chillee - [totally untested sym\_sqrt support](https://github.com/pytorch/pytorch/pull/84246/commits/fe4b196faa6536f6e8813b32585460ab008e1bbe) ezyang - [Get the magic method try reverse protocol correct](https://github.com/pytorch/pytorch/pull/84246/commits/62b3feabd66085556bfbb4d9c9412443d31a1c26) ezyang - [Fix accuracy minifier](https://github.com/pytorch/pytorch/pull/84246/commits/a49cc06a340db2076ffedef6e5f665deea293ef4) ezyang - [Fix another sympy infinite loop](https://github.com/pytorch/pytorch/pull/84246/commits/c0b9f783748657b7c895a6ce909993afa506cb8e) Chillee - [disable aot autograd cache](https://github.com/pytorch/pytorch/pull/84246/commits/7f82efeb1cfbc3a9707e1e8af3ff4c79c6673f38) anjali411 - [Properly pass on shape\_env to view base](https://github.com/pytorch/pytorch/pull/84246/commits/b5d018a44bad019e6bac63ee4d20edadd59880bd) ezyang - [Force people to call from\_meta\_and\_device directly](https://github.com/pytorch/pytorch/pull/84246/commits/2fc10ebfbb5ae24a7bdf32b92ebf576016f32503) ezyang - [Convert MetaConverter’s tensor memo into a weak value dictionary.](https://github.com/pytorch/pytorch/pull/84246/commits/2dc3070dd50aceea07c7fc7e1dda17d3f6b9da1d) ezyang - [Unify meta tensor and fake tensor converter conversion](https://github.com/pytorch/pytorch/pull/84246/commits/22b9ee8ab76811f30509d14af857d3073c2fce62) ezyang ## Merge to master retrospective - [Many symintifications](https://github.com/pytorch/pytorch/pull/87604) was reverted because it broke internal Executorch build, due to an Executorch only YAML file defining an out variant of an operator ([https://www.internalfb.com/diff/D40798763](https://www.internalfb.com/diff/D40798763)). The fbcode change ended up being trivial, so we relanded the PR by landing it GH1, and then ninja’ing the fbcode fix after it was imported in diff train. @albanD was initially concerned about merge conflicts between fbcode master and GH master because this was a large patch, but this strategy neatly sidestepped the problem (the biggest annoyance being ensuring the diff train was sufficiently imported to import this diff). - [Fix bernoulli functionalization.](https://github.com/pytorch/pytorch/pull/87573) required non-trivial XLA changes, but at time of writing @ezyang wasn’t able to compile XLA successfully. Fortunately, JackCaoG helped do the XLA side patch (which was relatively long, but not too difficult; just undoing XLA’s bernoulli hack.) - [Unify meta tensor and fake tensor converter conversion](https://github.com/pytorch/pytorch/pull/87943) was reverted on master because inductor tests were not run on PR. This will be fixed by enabling inductor CI on all fake tensor changes. ## What’s made it to master this week? - albanD (merge captain) - [Many symintifications](https://github.com/pytorch/pytorch/pull/87604) and [Reland 2 Many symintifications (#87604)](https://github.com/pytorch/pytorch/pull/87980) - [Fix a PyObject leak](https://github.com/pytorch/pytorch/pull/87608) - [Add /= to c10::SymInt](https://github.com/pytorch/pytorch/pull/87603) - [small improvement to error message in fx interpreter](https://github.com/pytorch/pytorch/pull/87599) - ezyang - [Unify SymIntNode and SymFloatNode into SymNode](https://github.com/pytorch/pytorch/pull/87817) + [Fix pybind11 problems with c10::SymInt unregistered](https://github.com/pytorch/pytorch/pull/88011) - [Convert MetaConverter’s tensor memo into a weak value dictionary.](https://github.com/pytorch/pytorch/pull/87911) - [Force people to call from\_meta\_and\_device directly](https://github.com/pytorch/pytorch/pull/87903) - [Fix bernoulli functionalization.](https://github.com/pytorch/pytorch/pull/87573) - [Improve argument printing](https://github.com/pytorch/pytorch/pull/87601) - [Fix accuracy minifier](https://github.com/pytorch/pytorch/pull/87606) - [as\_strided\_scatter storage offset defaults to None not 0](https://github.com/pytorch/pytorch/pull/87481) - bdhirsh - [add nesting to TORCH\_SHOW\_DISPATCH\_TRACE](https://github.com/pytorch/pytorch/pull/87751) - [functionalization: fix detach()](https://github.com/pytorch/pytorch/pull/87750) - [functionalization: make view\_copy outputs always contiguous](https://github.com/pytorch/pytorch/pull/85747) - anjali411 - [Remove custom Ceil in favor of sympy.ceiling](https://github.com/pytorch/pytorch/pull/87294) - Chillee - [fix sym\_storage conversion and some cleanup](https://github.com/pytorch/pytorch/pull/87718) - [Add get\_guard\_expr to symbolic\_shapes which returns all guards in a single expression](https://github.com/pytorch/pytorch/pull/87665) - voz - [[Dynamo] Symbolic shape guards](https://github.com/pytorch/pytorch/pull/87570) ## What’s coming next? - Educate the team on how torchdynamo dynamic shapes is supposed to work, and spend a lot of time fixing issues here - E2E training on master with inductor. - Hook up torchdynamo’s symbolic shapes to AOT Autograd’s symbolic shapes, check [Sym Shapes, Control Flow in Dynamo - RFC + Plan of Record - Google Docs](https://docs.google.com/document/d/1QJ-M4zfMkD-fjHIqW089RptjLl9EgozZGCceUbvmgfY/edit#heading=h.o9xfjd8yjybf) for design details) - Resolve strategy for sharing ShapeEnv between forward and backwards (@ezyang’s take: don’t return SymInts from forward pass) - Reshape unification - All benchmark models are passing aot\_eager training on branch; tracked at [Operators that need symbolic support - Google Sheets](https://docs.google.com/spreadsheets/d/1-ghURoD6tzTushbCb22tFd5B5uNg7HSxVG4dQBwoaFA/edit?pli=1#gid=401920381) - Fallback implementation for custom operators without symbolic shape propagation, inferred by running fallback on real operators - All OpInfo tests passing --- _[View the full topic](https://dev-discuss.pytorch.org/t/state-of-symbolic-shapes-branch/777)._