# Torch.compile can be debugged now!

**URL:** <https://dev-discuss.pytorch.org/t/torch-compile-can-be-debugged-now/1595>\
**Category:** compiler\
**Created:** [October 23, 2023, 3:42am UTC](https://dev-discuss.pytorch.org/t/torch-compile-can-be-debugged-now/1595 "2023-10-23T03:42:53Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![youkaichao](https://yyz2.discourse-cdn.com/flex036/user_avatar/dev-discuss.pytorch.org/youkaichao/32/1278_2.png) [@youkaichao](https://dev-discuss.pytorch.org/u/youkaichao)\
**Post date:** [October 23, 2023, 3:42am UTC](https://dev-discuss.pytorch.org/t/torch-compile-can-be-debugged-now/1595/1 "2023-10-23T03:42:53Z")

</div>

We are happy to announce that, with the newly introduced `depyf` package, we can debug `torch.compile` generated code now!

Example usage:

```python
import torch

@torch.compile(backend="eager")
def toy_example(a, b):
    x = a / (torch.abs(a) + 1)
    if b.sum() < 0:
        b = b * -1
    return x * b

import depyf
with depyf.prepare_debug(toy_example, "./dump_src_debug_function"):
    for _ in range(100):
        toy_example(torch.randn(10), torch.randn(10))

with depyf.debug():
    toy_example(torch.randn(10), torch.randn(10))

```

General guideline for debugging `torch.compile`:

- Use `backend="eager"` to debug Dynamo first, as this is the major place of black magic. If Dynamo works as you expect, it is straight-forward to debug the backend.
- Run your code under `with depyf.prepare_debug` context for enough times, so that all branches of your code get compiled.
- When leaving the `with depyf.prepare_debug` context, you will be prompted to set breakpoints for Dynamo generated code.
- The following code under `with depyf.debug():` context is debuggable.

And of course, remember to `pip install depyf` first.

A demo gif:

![output](https://canada1.discourse-cdn.com/flex036/uploads/pytorch1/original/2X/d/d848f97e32fcce726718be176a11956dfd565599.gif)

More information at [GitHub - thuml/depyf: Decompile python bytecode, and understand PyTorch compiler!](https://github.com/thuml/depyf) .

Hope it helps!

---

<div class="post-metadata">

**Author:** ![GLZhu](https://yyz2.discourse-cdn.com/flex036/user_avatar/dev-discuss.pytorch.org/glzhu/32/986_2.png) [@GLZhu](https://dev-discuss.pytorch.org/u/GLZhu)\
**Post date:** [October 24, 2023, 12:41am UTC](https://dev-discuss.pytorch.org/t/torch-compile-can-be-debugged-now/1595/2 "2023-10-24T00:41:23Z")

</div>

> [@youkaichao](#):
>
> he newly introduced

wow, much thank to team pytorch!  
One quick question, what if I use inductor rather than eager backend.

---

<div class="post-metadata">

**Author:** ![youkaichao](https://yyz2.discourse-cdn.com/flex036/user_avatar/dev-discuss.pytorch.org/youkaichao/32/1278_2.png) [@youkaichao](https://dev-discuss.pytorch.org/u/youkaichao)\
**Post date:** [October 24, 2023, 11:21am UTC](https://dev-discuss.pytorch.org/t/torch-compile-can-be-debugged-now/1595/3 "2023-10-24T11:21:59Z")

</div>

Dynamo and Inductor can be separatedly debugged. By using eager backend, you are eseentially debugging Dynamo only.