# Making PyTorch Test Suite usable from out-of-tree braindump

**URL:** <https://dev-discuss.pytorch.org/t/making-pytorch-test-suite-usable-from-out-of-tree-braindump/3357>\
**Category:** testing\
**Created:** [April 16, 2026, 6:36pm UTC](https://dev-discuss.pytorch.org/t/making-pytorch-test-suite-usable-from-out-of-tree-braindump/3357 "2026-04-16T18:36:07Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![mikaylagawarecki](https://yyz2.discourse-cdn.com/flex036/user_avatar/dev-discuss.pytorch.org/mikaylagawarecki/32/1124_2.png) [@mikaylagawarecki](https://dev-discuss.pytorch.org/u/mikaylagawarecki)\
**Post date:** [April 16, 2026, 6:36pm UTC](https://dev-discuss.pytorch.org/t/making-pytorch-test-suite-usable-from-out-of-tree-braindump/3357/1 "2026-04-16T18:36:07Z")

</div>

The purpose of this post is to brain-dump any context I had related to making the pytorch test suite device-generic as I am unfortunately moving teams out of PyTorch due to some reshuffling and will not be able to help drive this anymore 😔

Most of this information will not be new but the purpose is to consolidate all the context in one place.

This is what I think should have happened

1. Provide (a) a sample test runner in openreg (b) documentation that classifies tests based on feature based on [Device-Generic Refactoring Test Class Tracker](https://docs.google.com/spreadsheets/d/198SOBGz4tYXTosQdXWul4P2lUd0Dpw-Ql4njCylkThE/edit?gid=0#gid=0) and makes explicit which tests are legacy vs necessary to run to support a given feature

2. Provide a blessed mechanism for applying skips and decorators to tests from out of tree at all like how in-tree backends would. This applies at different granularities (e.g. OpInfo tests decorated with `@ops` vs for regular test methods).

3. Refactor relevant in-tree tests to run on device [Device-Generic Refactoring Test Class Tracker](https://docs.google.com/spreadsheets/d/198SOBGz4tYXTosQdXWul4P2lUd0Dpw-Ql4njCylkThE/edit?gid=0#gid=0). The above spreadsheet elucidates which test classes need to be refactored, and also attempts some priorities for the “generic” tab. Priority-wise I would have gone with generic → distributed → others.

4. Ensure that system persists

After all the above is tackled we could optionally tackle

1. Provide a registration mechanism for device-capabilities that is consumed by tests, annotate tests as appropriate (e.g. supported\_dtypes, supports\_memory\_format etc.) like [https://github.com/pytorch/pytorch/issues/146898](https://github.com/pytorch/pytorch/issues/146898)

---

<div class="post-metadata">

**Author:** ![fffrog](https://avatars.discourse-cdn.com/v4/letter/f/6de8d8/32.png) [@fffrog](https://dev-discuss.pytorch.org/u/fffrog)\
**Post date:** [April 27, 2026, 3:39am UTC](https://dev-discuss.pytorch.org/t/making-pytorch-test-suite-usable-from-out-of-tree-braindump/3357/2 "2026-04-27T03:39:17Z")

</div>

Hey @mikaylagawarecki , thank you for summarizing those great things, best wishes for your new role! 🙂