# Torch.compile with AOT Autograd can be debugged now!

**URL:** <https://dev-discuss.pytorch.org/t/torch-compile-with-aot-autograd-can-be-debugged-now/1624>\
**Category:** compiler\
**Created:** [October 31, 2023, 1:02pm UTC](https://dev-discuss.pytorch.org/t/torch-compile-with-aot-autograd-can-be-debugged-now/1624 "2023-10-31T13:02:37Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![youkaichao](https://yyz2.discourse-cdn.com/flex036/user_avatar/dev-discuss.pytorch.org/youkaichao/32/1278_2.png) [@youkaichao](https://dev-discuss.pytorch.org/u/youkaichao)\
**Post date:** [October 31, 2023, 1:02pm UTC](https://dev-discuss.pytorch.org/t/torch-compile-with-aot-autograd-can-be-debugged-now/1624/1 "2023-10-31T13:02:37Z")

</div>

Example code:

```python
from typing import List
import torch
from torch import _dynamo as torchdynamo
from depyf.explain.backend import eager, aot_eager

@torch.compile(backend=aot_eager)
def toy_example(a, b):
    x = a / (torch.abs(a) + 1)
    if b.sum() < 0:
        b = b * -1
    return x * b

import depyf
with depyf.prepare_debug(toy_example, "./debug_dir"):
    for _ in range(100):
        input = torch.randn(10, requires_grad=True), torch.randn(10, requires_grad=True)
        toy_example(*input)
with depyf.debug():
    input = torch.randn(10, requires_grad=True), torch.randn(10, requires_grad=True)
    toy_example(*input).sum().backward()

```

Example debug screenshot:

breakpoint at the `abs` operation in the forward pass of the first subgraph:

 ![image](https://canada1.discourse-cdn.com/flex036/uploads/pytorch1/original/2X/d/d96ee3519d48fc26175aab6fa17a8948486ecd76.png)

breakpoint at the grad of `abs` operation (i.e. the `sgn` function) in the backward pass of the first subgraph:

 ![image](https://canada1.discourse-cdn.com/flex036/uploads/pytorch1/original/2X/3/3ded7ac8532da230cfa7ab3d09af557219385b90.png)

For more details, please refer to the repo [GitHub - thuml/depyf: Decompile python bytecode, debug and understand PyTorch compiler!](https://github.com/thuml/depyf) .

Thanks for the helpful discussion from @Chillee and @bdhirsh in [How does torch.compile work with autograd?](https://dev-discuss.pytorch.org/t/how-does-torch-compile-work-with-autograd/1621) .

---

<div class="post-metadata">

**Author:** ![bdhirsh](https://yyz2.discourse-cdn.com/flex036/user_avatar/dev-discuss.pytorch.org/bdhirsh/32/575_2.png) [@bdhirsh](https://dev-discuss.pytorch.org/u/bdhirsh)\
**Post date:** [October 31, 2023, 2:24pm UTC](https://dev-discuss.pytorch.org/t/torch-compile-with-aot-autograd-can-be-debugged-now/1624/2 "2023-10-31T14:24:19Z")

</div>

This is very cool! 🙂