# Addressing Test hole for PrimTorch decomposition operators **URL:** \ **Category:** testing\ **Created:** [June 19, 2023, 10:27pm UTC](https://dev-discuss.pytorch.org/t/addressing-test-hole-for-primtorch-decomposition-operators/1336 "2023-06-19T22:27:44Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![ekamiti](https://yyz2.discourse-cdn.com/flex036/user_avatar/dev-discuss.pytorch.org/ekamiti/32/1113_2.png) [@ekamiti](https://dev-discuss.pytorch.org/u/ekamiti)\ **Post date:** [June 19, 2023, 10:27pm UTC](https://dev-discuss.pytorch.org/t/addressing-test-hole-for-primtorch-decomposition-operators/1336/1 "2023-06-19T22:27:44Z")
I just worked on an [issue](https://github.com/pytorch/pytorch/issues/100698) addressing mismatched behavior of an operator between compiled and eager modes. While implementing the test to verify the fix I noticed not all operator groupings in [common\_methods\_invocations.py](https://github.com/pytorch/pytorch/blob/13ef0ec186ef7f1230a62020bca0a7bf14bdc198/torch/testing/_internal/common_methods_invocations.py) include the ref operators: [https://github.com/pytorch/pytorch/blob/13ef0ec186ef7f1230a62020bca0a7bf14bdc198/torch/testing/\_internal/common\_methods\_invocations.py#L20599-L20614](https://github.com/pytorch/pytorch/blob/13ef0ec186ef7f1230a62020bca0a7bf14bdc198/torch/testing/_internal/common_methods_invocations.py#L20599-L20614) These groupings are used in some tests such as [test\_reductions](https://github.com/pytorch/pytorch/blob/13ef0ec186ef7f1230a62020bca0a7bf14bdc198/test/test_reductions.py). When I modified the test to run also for the ref operators there were some failures so this seems like a testing hole that may be allowing for some mismatched behavior between eager and compiled modes but maybe it’s intentional? I’d like to go through and fix some of these unless that’s something that has been looked at before but I couldn’t find any bugs about this but maybe I’m not looking in the right places. ---
**Author:** ![ezyang](https://yyz2.discourse-cdn.com/flex036/user_avatar/dev-discuss.pytorch.org/ezyang/32/12_2.png) [@ezyang](https://dev-discuss.pytorch.org/u/ezyang)\ **Post date:** [June 20, 2023, 8:35pm UTC](https://dev-discuss.pytorch.org/t/addressing-test-hole-for-primtorch-decomposition-operators/1336/2 "2023-06-20T20:35:15Z")
I think we’d definitely appreciate fixes along these lines. There isn’t really anyone captaining the ship here, so you may need to figure out precisely what the right strategy is. ---
**Author:** ![ekamiti](https://yyz2.discourse-cdn.com/flex036/user_avatar/dev-discuss.pytorch.org/ekamiti/32/1113_2.png) [@ekamiti](https://dev-discuss.pytorch.org/u/ekamiti)\ **Post date:** [June 20, 2023, 10:30pm UTC](https://dev-discuss.pytorch.org/t/addressing-test-hole-for-primtorch-decomposition-operators/1336/3 "2023-06-20T22:30:12Z")
Sounds good. I’ll catalog the failures and then go from there. ---
**Author:** ![ekamiti](https://yyz2.discourse-cdn.com/flex036/user_avatar/dev-discuss.pytorch.org/ekamiti/32/1113_2.png) [@ekamiti](https://dev-discuss.pytorch.org/u/ekamiti)\ **Post date:** [July 25, 2023, 11:42pm UTC](https://dev-discuss.pytorch.org/t/addressing-test-hole-for-primtorch-decomposition-operators/1336/4 "2023-07-25T23:42:30Z")
So finally finished going through the first batch of failures when getting some of the tests to run for `_ref` operators as they do for the `eager` mode ones. Good news is that it seems most of the failures are just due to inherited attributes in the test definitions that resulted in some tests not being skipped for the `_ref` versions as they are in the `eager` version. There is one issue that seems to be a legitimate issue and filed issue [Inconsistent behavior for \_refs.fft.\* operators when input is an empty tensor · Issue #105986 · pytorch/pytorch · GitHub](https://github.com/pytorch/pytorch/issues/105986). Hoping to get some more test results and see what else needs to be addressed.