# PyTorch 1.11 dev release notes

**URL:** <https://dev-discuss.pytorch.org/t/pytorch-1-11-dev-release-notes/588>\
**Category:** release/packaging\
**Created:** [April 21, 2022, 6:23pm UTC](https://dev-discuss.pytorch.org/t/pytorch-1-11-dev-release-notes/588 "2022-04-21T18:23:02Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![bdhirsh](https://yyz2.discourse-cdn.com/flex036/user_avatar/dev-discuss.pytorch.org/bdhirsh/32/575_2.png) [@bdhirsh](https://dev-discuss.pytorch.org/u/bdhirsh)\
**Post date:** [April 21, 2022, 6:23pm UTC](https://dev-discuss.pytorch.org/t/pytorch-1-11-dev-release-notes/588/1 "2022-04-21T18:23:02Z")

</div>

A bit delayed, but - we have quite a few commits in the 1.11 release and some things that are interesting for people that develop within PyTorch.  
You can find below a curated list of these changes:

# Developers

## Python API

- OpInfo improvements:
  - More operators now have OpInfo tests:
    - Added `OpInfo` for `nn.functional.batch_norm` ([#63218](https://github.com/pytorch/pytorch/pull/63218)),
    - Added `OpInfo` for `torch.argsort` ([#65454](https://github.com/pytorch/pytorch/pull/65454))
    - Added `OpInfo` for `torch.repeat_interleave` ([#65455](https://github.com/pytorch/pytorch/pull/65455))
    - Added `OpInfo` for `2d fft functions` ([#66128](https://github.com/pytorch/pytorch/pull/66128))
    - Added `Opinfo`’s for `avg_pooling` ([#64214](https://github.com/pytorch/pytorch/pull/64214))
    - Added `OpInfo `for `torch.bucketize` ([#65821](https://github.com/pytorch/pytorch/pull/65821))
    - Added `OpInfo`’s for `isfinite`, `isinf`, `isposinf`, `isneginf`, `isnan`, `isreal` ([#66400](https://github.com/pytorch/pytorch/pull/66400))
    - Added `OpInfo` for `torch.nn.functional.pairwise_distance` ([#65460](https://github.com/pytorch/pytorch/pull/65460))
    - Added `OpInfo` for `torch.nn.pixel_shuffle` ([#65467](https://github.com/pytorch/pytorch/pull/65467))
    - Added `OpInfo` for `torch.nn.pixel_unshuffle` ([#65468](https://github.com/pytorch/pytorch/pull/65468))
    - Added `OpInfo` for `torch.bincount `([#65796](https://github.com/pytorch/pytorch/pull/65796))
    - Added `OpInfo` for `norm` ops ([#67442](https://github.com/pytorch/pytorch/pull/67442), [#68526](https://github.com/pytorch/pytorch/pull/68526))
    - Added `OpInfo` for `torch.nn.functional.gaussian_nll_loss` ([#67356](https://github.com/pytorch/pytorch/pull/67356))
    - Added `OpInfo` for `nn.functional.hinge_embedding_loss` ([#67381](https://github.com/pytorch/pytorch/pull/67381))
    - Added `OpInfo` for `nn.functional.gaussian_nll_loss` ([#67376](https://github.com/pytorch/pytorch/pull/67376))
    - Added `OpInfo` for `nn.functional.poisson_nll_loss` ([#67371](https://github.com/pytorch/pytorch/pull/67371))
    - Added `OpInfo` for `nn.functional.ctc_loss` ([#67464](https://github.com/pytorch/pytorch/pull/67464))
    - Added `OpInfo` for `nn.functional.cosine_embedding_loss` ([#67465](https://github.com/pytorch/pytorch/pull/67465))
    - Added `OpInfo` for `adaptive_max_pool` ([#67405](https://github.com/pytorch/pytorch/pull/67405))
    - Added `OpInfo` for `logical_or`, `logical_and`, `logical_xor` ([#67178](https://github.com/pytorch/pytorch/pull/67178))
    - Added `OpInfo` for `torch.allclose` ([#68023](https://github.com/pytorch/pytorch/pull/68023))
    - Added `OpInfo` for `nn.functional.cross_entropy` ([#63547](https://github.com/pytorch/pytorch/pull/63547))
    - Added `OpInfo` for `torch.nn.bilinear` and `torch.nn.glu` ([#67478](https://github.com/pytorch/pytorch/pull/67478))
    - Added `OpInfo` for `torch.histc` ([#67452](https://github.com/pytorch/pytorch/pull/67452))
    - Added `OpInfos` for `stft, istft, fftshift, ifftshift` ([#68198](https://github.com/pytorch/pytorch/pull/68198))
    - Added `OpInfos` for `parcel Elementwise Binary II` ([#68085](https://github.com/pytorch/pytorch/pull/68085))
    - Added `OpInfo` for `torch.linalg.tensorsolve` ([#68810](https://github.com/pytorch/pytorch/pull/68810))
    - Added `OpInfo` for `torch.nn.functional.kl_div` ([#65469](https://github.com/pytorch/pytorch/pull/65469))
    - Added `OpInfo` for `torch.diagflat` ([#65680](https://github.com/pytorch/pytorch/pull/65680))
    - Added `OpInfo`s for some Tensor dtype conversion methods ([#64282](https://github.com/pytorch/pytorch/pull/64282))
    - Added `OpInfo` for `*_like` functions ([#65941](https://github.com/pytorch/pytorch/pull/65941))
    - Added `OpInfo` for `torch.unique` and `torch.unique_consecutive` ([#67529](https://github.com/pytorch/pytorch/pull/67529))
    - Added `OpInfo` for `new_` functions and some `_like` functions ([#67357](https://github.com/pytorch/pytorch/pull/67357))
    - Added `OpInfo` for `torch.nonzero` ([#67459](https://github.com/pytorch/pytorch/pull/67459))
    - Added `OpInfos` for `torch.atleast_`{1d, 2d, 3d} ([#67355](https://github.com/pytorch/pytorch/pull/67355))
    - Added `OpInfo` for `embedding_bag` ([#67252](https://github.com/pytorch/pytorch/pull/67252))
    - Added `OpInfos` for `combinations`, `cartesian_prod`, `sum_to_size`, `ldexp`, and `as_stride`d ([#68853](https://github.com/pytorch/pytorch/pull/68853))
    - Added `OpInfos` for misc `nn.functional` operators ([#68922](https://github.com/pytorch/pytorch/pull/68922))
    - Added `OpInfo` tests for `(svd|pca)_lowrank` ([#69107](https://github.com/pytorch/pytorch/pull/69107))
    - Added `OpInfo` for `nn.functional.dropout2d`, revise sample inputs for `dropout` ([#67891](https://github.com/pytorch/pytorch/pull/67891))
    - Added `OpInfos` for `normal`, `bernoulli`, `multinomial` ([#66358](https://github.com/pytorch/pytorch/pull/66358))
    - Added `OpInfos` for `flatten`, `column_stack` ([#69237](https://github.com/pytorch/pytorch/pull/69237))

  - Other improvements to `OpInfo` testing:
    - Added inplace\_variant for resize\_ `OpInfo` ([#66135](https://github.com/pytorch/pytorch/pull/66135))
    - Added reference vs. noncontiguous `OpInfo` test ([#67434](https://github.com/pytorch/pytorch/pull/67434))
    - Split channels\_last test cases for tensor conversion `OpInfos` ([#67368](https://github.com/pytorch/pytorch/pull/67368))
    - Remove `OpInfo` non-contig inputs ([#67677](https://github.com/pytorch/pytorch/pull/67677))
    - Improve `OpInfo` test for norm ops: make inputs independent
    - [opinfo] use dtypes instead of `dtypesIfCPU` ([#68732](https://github.com/pytorch/pytorch/pull/68732))
    - Fix for python 3.10 for gradient `Opinfo `([#68113](https://github.com/pytorch/pytorch/pull/68113))
    - `OpInfo`: Convert more `sample_input_funcs` to generators ([#69976](https://github.com/pytorch/pytorch/pull/69976))
    - Updated `poisson_nll_loss` `Opinfo` samples ([#70300](https://github.com/pytorch/pytorch/pull/70300))
    - Removed unnecessary skips in rsub `OpInfo` ([#69973](https://github.com/pytorch/pytorch/pull/69973))
    - Merged index\_{add,fill,copy,select} `OpInfo` sampling ([#68184](https://github.com/pytorch/pytorch/pull/68184))
    - Labeled more elementwise binary operators correctly as `BinaryUfuncInfos` ([#71622](https://github.com/pytorch/pytorch/pull/71622))
    - Deactivated the tracking of gradients in sampling functions within `OpInfos` ([#68522](https://github.com/pytorch/pytorch/pull/68522))
    - Removed special FX `OpInfo` list ([#67520](https://github.com/pytorch/pytorch/pull/67520))

- More informative messages for None types comparisons ([#69802](https://github.com/pytorch/pytorch/pull/69802))
- Killed the `test_torch.py` mixin and created test\_scatter\_gather\_ops ([#71691](https://github.com/pytorch/pytorch/pull/71691))
- Relaxes tolerance on ROCm `test_noncontiguous_samples_matmul` ([#67593](https://github.com/pytorch/pytorch/pull/67593))
- Added support for automated error and warning testing ([#67354](https://github.com/pytorch/pytorch/pull/67354))
- Skip forward-over-reverse gradgrad check for pinv singular on CUDA ([#70123](https://github.com/pytorch/pytorch/pull/70123))
- Made meta tensor data access error message for expressive in `assert_close` ([#68802](https://github.com/pytorch/pytorch/pull/68802))
- Removed skips from determinant tests ([#70034](https://github.com/pytorch/pytorch/pull/70034))
- Refactored repetitions into `TorchVersion._cmp_wrapper` ([#71344](https://github.com/pytorch/pytorch/pull/71344))
- Expect `test_fn_fwgrad_bwgrad` to fail because forward AD is not implemented ([#71944](https://github.com/pytorch/pytorch/pull/71944))
- Some python tensor subclass improvements:
  - Added Tensor.\_make\_wrapper\_subclass ([#65340](https://github.com/pytorch/pytorch/pull/65340))
  - **getitem** : Ensure Tensor subclasses are not treated as tuples ([#67202](https://github.com/pytorch/pytorch/pull/67202))
  - Fixed `_make_wrapper_subclass`’s storage\_offset handling ([#68268](https://github.com/pytorch/pytorch/pull/68268))
  - Make empty \*\*_and_ \*\* \_like factory functions respect tensor subclasses ([#65677](https://github.com/pytorch/pytorch/pull/65677))
  - Make new\_empty/new\_ones/new\_zeros/new\_full respect subclass ([#65169](https://github.com/pytorch/pytorch/pull/65169))
  - Ensure that “None” tensors in python map to “undefined” tensors in C++ ([#67793](https://github.com/pytorch/pytorch/pull/67793/files))

- Rationalized API exports in torch\_python ([#68095](https://github.com/pytorch/pytorch/pull/68095))
- Removed `tensor.data` usage from a few places in internals ([#65389](https://github.com/pytorch/pytorch/pull/65389))

## C++ API

- Convolution consolidation:
  - Factored backend routing logic out of convolution forward ([#67790](https://github.com/pytorch/pytorch/pull/67790))
  - General convolution\_backward function ([#69044](https://github.com/pytorch/pytorch/pull/69044), [#70112](https://github.com/pytorch/pytorch/pull/70112), [#71489](https://github.com/pytorch/pytorch/pull/71489), [#71490](https://github.com/pytorch/pytorch/pull/71490), [#71491](https://github.com/pytorch/pytorch/pull/71491), [#69584](https://github.com/pytorch/pytorch/pull/69584), [#67283](https://github.com/pytorch/pytorch/pull/67283), [#70661](https://github.com/pytorch/pytorch/pull/70661))
  - Removed finput, fgrad\_input, columns, and ones from slow{2,3}d and slow{2,3}d\_transpose signatures ([#68897](https://github.com/pytorch/pytorch/pull/68897), [#68898](https://github.com/pytorch/pytorch/pull/68898), [#68899](https://github.com/pytorch/pytorch/pull/68899))
  - Removed backward ops for: cuDNN convolution, cuDNN transposed convolution, deprecated cuDNN convolution, miopen convolution, miopen convolution, miopen transposed convolution, miopen depthwise convolution, slow dilated 2d convolution, slow 2d transposed convolution, slow 3d convolution, slow dilated 3d convolution, mkldnn convolution, low 3d transposed convolution, 2d depthwise convolution, 3d depthwise convolution, NNPACK spatial convolution ([#69901](https://github.com/pytorch/pytorch/pull/69901), [#69902](https://github.com/pytorch/pytorch/pull/69902), [#71128](https://github.com/pytorch/pytorch/pull/71128), [#69987](https://github.com/pytorch/pytorch/pull/69987), [#69987](https://github.com/pytorch/pytorch/pull/69987), [#70063](https://github.com/pytorch/pytorch/pull/70063), [#70064](https://github.com/pytorch/pytorch/pull/70064), [#70067](https://github.com/pytorch/pytorch/pull/70067), [#70333](https://github.com/pytorch/pytorch/pull/70333), [#69978](https://github.com/pytorch/pytorch/pull/69978), [#70068](https://github.com/pytorch/pytorch/pull/70068), [#70467](https://github.com/pytorch/pytorch/pull/70467), [#69933](https://github.com/pytorch/pytorch/pull/69933), [#70461](https://github.com/pytorch/pytorch/pull/70461),[#69902](https://github.com/pytorch/pytorch/pull/69902), [#70462](https://github.com/pytorch/pytorch/pull/70462), [#70305](https://github.com/pytorch/pytorch/pull/70305))

- Removed TH/THC logic ([#68127](https://github.com/pytorch/pytorch/pull/68127), [#68556](https://github.com/pytorch/pytorch/pull/68556), [#69040](https://github.com/pytorch/pytorch/pull/69040), [#69041](https://github.com/pytorch/pytorch/pull/69041), [#65942](https://github.com/pytorch/pytorch/pull/65942), [#69929](https://github.com/pytorch/pytorch/pull/69929), [#67940](https://github.com/pytorch/pytorch/pull/67940))
- Added tanh\_backward to AT symbols ([#70071](https://github.com/pytorch/pytorch/pull/70071))
- Improved documentation of comparison internals ([#68977](https://github.com/pytorch/pytorch/pull/68977))
- Added isUndefined to ExclusivelyOwnedTraits debug msg ([#70638](https://github.com/pytorch/pytorch/pull/70638))
- Removed buggy ExclusivelyOwnedTraits\> ([#70647](https://github.com/pytorch/pytorch/pull/70647))
- Generated aten\_interned\_strings.h automatically ([#69407](https://github.com/pytorch/pytorch/pull/69407))
- Empty\_strided: Factor out generic implementation ([#70614](https://github.com/pytorch/pytorch/pull/70614))
- Empty\_meta: Add functions that don’t depend on Tensor ([#70615](https://github.com/pytorch/pytorch/pull/70615))
- Consolidated the overloads of TensorImpl::shallow\_copy\_and\_detach ([#68953](https://github.com/pytorch/pytorch/pull/68953))
- Improved storage assertion of Tensor’s enforce\_invariants ([#70380](https://github.com/pytorch/pytorch/pull/70380))
- Fixed aten’s native’s folder docs. ([#71395](https://github.com/pytorch/pytorch/pull/71395))
- Use of new\_empty in dropout ([#72078](https://github.com/pytorch/pytorch/pull/72078))
- Simplified TensorImpl size check and fix error message ([#72070](https://github.com/pytorch/pytorch/pull/72070))
- Added output\_mask argument to `grid_sampler_2d_backward` ([#66068](https://github.com/pytorch/pytorch/pull/66068))
- Avoided no-op shared\_ptr dtor when constructing tuple ([#69337](https://github.com/pytorch/pytorch/pull/69337))
- slow\_conv2d grad\_weight: call gemm directly ([#65726](https://github.com/pytorch/pytorch/pull/65726))
- Made handle\_torch\_function\_no\_python\_arg\_parser public ([#66054](https://github.com/pytorch/pytorch/pull/66054))
- slow\_conv3d: Avoided dispatch in parallel region ([#65737](https://github.com/pytorch/pytorch/pull/65737))
- slow\_conv3d grad\_input: Avoided dispatch in parallel region ([#65757](https://github.com/pytorch/pytorch/pull/65757))
- slow\_conv3d: Used at::sum for grad\_bias accumulation ([#65758](https://github.com/pytorch/pytorch/pull/65758))
- TBB: Use static partitioner to match OpenMP scheduling ([#65327](https://github.com/pytorch/pytorch/pull/65327))
- Move intraop\_launch\_future from Parallel.h ([#64166](https://github.com/pytorch/pytorch/pull/64166))
- slow\_conv3d grad\_weight: call gemm directly ([#65759](https://github.com/pytorch/pytorch/pull/65759))
- Wextra fix for Tensorshape.cpp ([#66320](https://github.com/pytorch/pytorch/pull/66320))
- Add InplaceOrView boxed kernel ([#63878](https://github.com/pytorch/pytorch/pull/63878))
- Used `at::native::is_nonzero` in a few places to skip an unnecessary dispatch trip ([#67195](https://github.com/pytorch/pytorch/pull/67195))
- Added tags for inplace view ops in native\_functions.yaml ([#65412](https://github.com/pytorch/pytorch/pull/65412))
- Fixed C++ BatchNorm pretty\_print() with optional momentum ([#67335](https://github.com/pytorch/pytorch/pull/67335))
- Inserted check for PyObject\_IsInstance in THPVariableCheck ([#67588](https://github.com/pytorch/pytorch/pull/67588))
- Added SiLU backward Aten symbol ([#67665](https://github.com/pytorch/pytorch/pull/67665))
- Bumped dlpack.h to latest version ([#65047](https://github.com/pytorch/pytorch/pull/65047))
- Remove dWindowsTorchApiMacro.h in favor of Export.h ([#69585](https://github.com/pytorch/pytorch/pull/69585))
- Added macro to register CPU kernel for all arch types ([#70332](https://github.com/pytorch/pytorch/pull/70332))
- `c10::irange` around the codebase instead of for loops ([#70326](https://github.com/pytorch/pytorch/pull/70326))

## Autograd

- Forward AD can be tested in gradcheck and OpInfos without also testing backward AD ([#65040](https://github.com/pytorch/pytorch/pull/65040))
- Extended OpInfo and gradgradcheck to test forward-over-reverse Hessian-vector products ([#69740](https://github.com/pytorch/pytorch/pull/69740))
- Extended OpInfo and gradcheck to test batched forward grad ([#66294](https://github.com/pytorch/pytorch/pull/66294))
- Enabled warning tests for nondeterministic backward functions ([#66736](https://github.com/pytorch/pytorch/pull/66736))
- Extended autograd functional benchmarking to run vectorized tasks ([#67045](https://github.com/pytorch/pytorch/pull/67045))
- Disallowed requires\_grad=True in OpInfo’s `make_tensor` function for integral inputs ([#67149](https://github.com/pytorch/pytorch/pull/67149))
- Made autograd codegen for differentiable outputs safer to use ([#65823](https://github.com/pytorch/pytorch/pull/65823))

## Build

- Improved disable name match ([#71499](https://github.com/pytorch/pytorch/pull/71499))
- Made permission errors more human readable when using setup.py ([#66492](https://github.com/pytorch/pytorch/pull/66492))

## torch.nn

- Added testing across `memory_format` types to `ModuleInfos` ([#69317](https://github.com/pytorch/pytorch/pull/69317))
- Added private `_masked_softmax` function ([#69268](https://github.com/pytorch/pytorch/pull/69268), [#69272](https://github.com/pytorch/pytorch/pull/69272), [#69924](https://github.com/pytorch/pytorch/pull/69924))
- Added `native_dropout` ([#63937](https://github.com/pytorch/pytorch/pull/63937))
- `F.interpolate`: Removed JIT FC tweaks for `antialias` flag and `nearest-exact` mode ([#71937](https://github.com/pytorch/pytorch/pull/71937))
- `F.pad`: Replaced `empty()` with `new_empty()` ([#68565](https://github.com/pytorch/pytorch/pull/68565))
- `F.softmax`: Changed `dtype` to support TorchScript and MyPy ([#68336](https://github.com/pytorch/pytorch/pull/68336))
- `nn.BatchNorm*d`: Incremented `num_batches_tracked` in place for improved graph safety ([#70444](https://github.com/pytorch/pytorch/pull/70444))
- `nn.Embedding`: Passed arguments of embedding as named arguments ([#67574](https://github.com/pytorch/pytorch/pull/67574))
- `nn.FractionalMaxPool2d`: Fixed to index correct `_random_samples` dimension when provided ([#70031](https://github.com/pytorch/pytorch/pull/70031))
- `nn.{GRU, LSTM, RNN}`: Fixed links to docs in comments ([#68828](https://github.com/pytorch/pytorch/pull/68828))
- `nn.Module`: Added private `_stateless` API ([#61447](https://github.com/pytorch/pytorch/pull/61447), [#68969](https://github.com/pytorch/pytorch/pull/68969))
- `nn.modules.utils.{_single,_pair,_triple,_quadruple}`: Populated ` __name__ ` ([#70459](https://github.com/pytorch/pytorch/pull/70459))
- `nn.Parameter`: Used `torch.empty()` instead of `torch.tensor()` ([#66486](https://github.com/pytorch/pytorch/pull/66486))
- `optim`: Updated `CODEOWNERS` ([#65773](https://github.com/pytorch/pytorch/pull/65773))
- `optim.Optimizer`: Integrated `multi_tensor` `zero_grad` into base class ([#69936](https://github.com/pytorch/pytorch/pull/69936))
- Refactored cuDNN convolution memory format and conv-bias-relu code ([#65594](https://github.com/pytorch/pytorch/pull/65594))
- Testing
  - Set cuDNN deterministic flag for `test_conv_double_backward_cuda` ([#69941](https://github.com/pytorch/pytorch/pull/69941))
  - Increased tolerance for `test_adadelta` ([#69919](https://github.com/pytorch/pytorch/pull/69919))
  - Set test owner for nn tests ([#66850](https://github.com/pytorch/pytorch/pull/66850))
  - Changed `test_conv_large` parameter initialization ([#71521](https://github.com/pytorch/pytorch/pull/71521))
  - Obliviated `ALL_TENSORTYPES` and `ALL_TENSORTYPES2` ([#71153](https://github.com/pytorch/pytorch/pull/71153))
  - Removed repeat test for types in `test_nn.py` ([#70872](https://github.com/pytorch/pytorch/pull/70872))
  - Tweaked `rel_tol` for `test_adadelta` ([#71880](https://github.com/pytorch/pytorch/pull/71880))
  - Added no-input-grad-needed cases to `test_grid_sample` ([#66071](https://github.com/pytorch/pytorch/pull/66071))
  - Added OpInfo entries for `nn.functional.{conv1d, linear}` ([#67747](https://github.com/pytorch/pytorch/pull/67747), [#65498](https://github.com/pytorch/pytorch/pull/65498))
  - Added host-side memory requirement for `test_softmax_64bit_indexing` ([#67922](https://github.com/pytorch/pytorch/pull/67922))
  - Made `@dtypes` mandatory when using `@dtypesIf` ([#68186](https://github.com/pytorch/pytorch/pull/68186))
  - Added testing for complex non-vanilla SGD ([#66261](https://github.com/pytorch/pytorch/pull/66261))
  - Skipped failing tests in `test_nn.py` if compiled without LAPACK ([#70913](https://github.com/pytorch/pytorch/pull/70913))

## torch.fx

- Supported type annotations in `operator_support.py` ([#65136](https://github.com/pytorch/pytorch/pull/65136))
- Added algo recorder/replayer to `lower.py` ([#68194](https://github.com/pytorch/pytorch/pull/68194))
- Traced asserts with fx by looking at bytecode ([#70960](https://github.com/pytorch/pytorch/pull/70960))
- Fixed type checking errors in node.py ([#68124](https://github.com/pytorch/pytorch/pull/68124))

## AMD

- Updated ROCm build to avoid relying on `CUDA_VERSION` or `HIP_VERSION` macros ([#65610](https://github.com/pytorch/pytorch/pull/65610))

---

<div class="post-metadata">

**Author:** ![bdhirsh](https://yyz2.discourse-cdn.com/flex036/user_avatar/dev-discuss.pytorch.org/bdhirsh/32/575_2.png) [@bdhirsh](https://dev-discuss.pytorch.org/u/bdhirsh)\
**Post date:** [April 21, 2022, 6:23pm UTC](https://dev-discuss.pytorch.org/t/pytorch-1-11-dev-release-notes/588/2 "2022-04-21T18:23:15Z")

</div>

(Continued)

## CUDA

- Moved ATen/CUDAGeneratorImpl.h to ATen/cuda ([#71224](https://github.com/pytorch/pytorch/pull/71224))
- empty\_cuda: Added functions that don’t depend on Tensor ([#70616](https://github.com/pytorch/pytorch/pull/70616))
- Minor ScanKernels.cu cleanup ([#65350](https://github.com/pytorch/pytorch/pull/65350))
- Removed THC ScalarConvert ([#65471](https://github.com/pytorch/pytorch/pull/65471))
- Removed THCTensor.cu and THCTensorCopy.cu copy ([#65491](https://github.com/pytorch/pytorch/pull/65491))
- Removed THCDeviceTensor ([#65744](https://github.com/pytorch/pytorch/pull/65744))
- Migrated THCIntegerDivider.cuh to ATen ([#65745](https://github.com/pytorch/pytorch/pull/65745))
- Added workaround for nvcc header dependecies bug ([#62550](https://github.com/pytorch/pytorch/pull/62550))
- Removed accscalar from i0 and i0e ([#67048](https://github.com/pytorch/pytorch/pull/67048))
- Moved some cub templates out of the header file ([#67650](https://github.com/pytorch/pytorch/pull/67650))
- Exposed more CUDA/CuDNN info to at::Context and BC stage 1 ([#68146](https://github.com/pytorch/pytorch/pull/68146))
- Added ModuleInfo-based CPU / GPU parity tests ([#68097](https://github.com/pytorch/pytorch/pull/68097))
- Added ModuleInfo-based device transfer tests ([#68092](https://github.com/pytorch/pytorch/pull/68092))
- Updated CUDA memory leak check to verify against driver API and print more diagnostic information ([#69556](https://github.com/pytorch/pytorch/pull/69556))
- Fixed build on latest main branch of thrust ([#69985](https://github.com/pytorch/pytorch/pull/69985))
- Split cuda: list cpp files that go in \_cu library explicitly ([#69082](https://github.com/pytorch/pytorch/pull/69082))

## Dispatcher

- Made `detach` re-dispatch like a regular PyTorch operator ([#71707](https://github.com/pytorch/pytorch/pull/71707))
- `index_backward`: used out-of-place index\_put if any input is subclass ([#71779](https://github.com/pytorch/pytorch/pull/71779))
- Some bug fixes to the external codegen pipeline to make it easier for external backends to use it ([#69950](https://github.com/pytorch/pytorch/pull/69950), [#69951](https://github.com/pytorch/pytorch/pull/69951), [#69949](https://github.com/pytorch/pytorch/pull/69949))
- Made several ops that are implemented as composite in C++ “compliant”: before they would not play well with custom tensor subclasses, and now they should. Testing logic added in ([#65819](https://github.com/pytorch/pytorch/pull/65819))
  - binary\_cross\_entropy backward ([#70198](https://github.com/pytorch/pytorch/pull/70198))
  - quantile and nanquantile ([#70894](https://github.com/pytorch/pytorch/pull/70894))
  - linalg.{matrix\_power, inv, cholesky} ([#69437](https://github.com/pytorch/pytorch/pull/69437))
  - index\_copy, index\_fill, masked\_scatter, masked\_fill ([#71751](https://github.com/pytorch/pytorch/pull/71751))
  - index\_put ([#71765](https://github.com/pytorch/pytorch/pull/71765))
  - gather\_backward ([#71766](https://github.com/pytorch/pytorch/pull/71766))

## Mobile

- Split the upgrader test to a separate file and cover mobile part ([#70090](https://github.com/pytorch/pytorch/pull/70090))
- Removed unused variable in applyUpgrader ([#70261](https://github.com/pytorch/pytorch/pull/70261))
- Better error message when training attribute is not found ([#68103](https://github.com/pytorch/pytorch/pull/68103))
- Disabled miopen test for convolution on mobile ([#66564](https://github.com/pytorch/pytorch/pull/66564))
- Bumped up iOS CocoaPods version to 1.10.0 ([#67058](https://github.com/pytorch/pytorch/pull/67058))
- Lite interpreter naming for android nightly publishing ([#68651](https://github.com/pytorch/pytorch/pull/68651))
- Set test owner for mobile tests ([#66829](https://github.com/pytorch/pytorch/pull/66829))
- Added ownership to more edge tests ([#67859](https://github.com/pytorch/pytorch/pull/67859))
- Skipped compiledWithCuDNN() call for mobile to avoid segfault ([#71775](https://github.com/pytorch/pytorch/pull/71775))
- System specific adjustments for UTs to work. ([#65245](https://github.com/pytorch/pytorch/pull/65245))
- Updated mobile observer API for inference metadata logging ([#65451](https://github.com/pytorch/pytorch/pull/65451))
- Made the error message of missing ops to be more specific ([#71294](https://github.com/pytorch/pytorch/pull/71294))
- Exposed is\_metal\_available in header ([#68942](https://github.com/pytorch/pytorch/pull/68942))
- Removed unused function in import ([#65865](https://github.com/pytorch/pytorch/pull/65865))
- TensorExprKernel: support custom-class constants ([#68856](https://github.com/pytorch/pytorch/pull/68856))
- Moved all serialize/deserialize files to a separate target ([#66805](https://github.com/pytorch/pytorch/pull/66805))
- Added Backport test ([#67824](https://github.com/pytorch/pytorch/pull/67824))
- Exposed methods and compilation unit ([#66854](https://github.com/pytorch/pytorch/pull/66854))
- Populated operator\_input\_sizes ([#68542](https://github.com/pytorch/pytorch/pull/68542))
- Updated generated header to use flatbuffer v1.12 ([#71279](https://github.com/pytorch/pytorch/pull/71279))
- Refactored flatbuffer loader to allow overriding how IValues are parsed ([#71661](https://github.com/pytorch/pytorch/pull/71661))
- Removed StringView from RecordFunction interface (1/2) ([#68410](https://github.com/pytorch/pytorch/pull/68410))
- Moved upgraders from python to cpp ([#70593](https://github.com/pytorch/pytorch/pull/70593))
- Moved bytecode generation to python ([#71681](https://github.com/pytorch/pytorch/pull/71681))
- Made upgrader test model generation more robust ([#72030](https://github.com/pytorch/pytorch/pull/72030))
- Created convinience wrapper for dynamic type construcytors ([#71457](https://github.com/pytorch/pytorch/pull/71457))
- Enabled upgraders in TS server ([#70539](https://github.com/pytorch/pytorch/pull/70539))
- Added a helper to produce html with a single call in model\_dump ([#66005](https://github.com/pytorch/pytorch/pull/66005))
- Skipped writing version during backport ([#65842](https://github.com/pytorch/pytorch/pull/65842))
- Moved TypeParser class definition to header file ([#65976](https://github.com/pytorch/pytorch/pull/65976))
- Updated bytecode version compatibility check ([#67417](https://github.com/pytorch/pytorch/pull/67417))
- Added complete type name in error message when fail to export model ([#67750](https://github.com/pytorch/pytorch/pull/67750))
- Added old models and unittest ([#67726](https://github.com/pytorch/pytorch/pull/67726))
- Updated upgrader codegen with latest change ([#70293](https://github.com/pytorch/pytorch/pull/70293))
- Used hypothesis for better test input data and broader coverage ([#70263](https://github.com/pytorch/pytorch/pull/70263))
- Removed version compare as they are decoupled now ([#71461](https://github.com/pytorch/pytorch/pull/71461))
- Automated model generating process ([#70629](https://github.com/pytorch/pytorch/pull/70629))
- Moved generated keyword out of gen\_mobile\_upgraders.py ([#71938](https://github.com/pytorch/pytorch/pull/71938))
- Used upgrader\_mobile.cpp as the reference for codegen unittest ([#71930](https://github.com/pytorch/pytorch/pull/71930))
- Added type check in compatibility api ([#63129](https://github.com/pytorch/pytorch/pull/63129))
- Promoted missing ops for delegated models ([#66052](https://github.com/pytorch/pytorch/pull/66052))
- Used at::native::is\_nonzero in promoted ops to improve portability ([#67097](https://github.com/pytorch/pytorch/pull/67097))
- Set actual output type, remove ambiguity from compile\_spec names ([#67209](https://github.com/pytorch/pytorch/pull/67209))
- Set kernel func name from compiler ([#67229](https://github.com/pytorch/pytorch/pull/67229))
- Used irange for loops ([#66747](https://github.com/pytorch/pytorch/pull/66747))
- Added control stack frame to lite interpreter ([#65963](https://github.com/pytorch/pytorch/pull/65963))
- Implemented torch::jit::Function for mobile funciton ([#65970](https://github.com/pytorch/pytorch/pull/65970))
- Loaded interface methods to corresponding ClassTypes ([#65971](https://github.com/pytorch/pytorch/pull/65971))
- Removed usage of shared\_ptr ([#68037](https://github.com/pytorch/pytorch/pull/68037))
- Created DynamicType for OptionalType in mobile ([#68137](https://github.com/pytorch/pytorch/pull/68137))
- Polymorphic IValue::type() for DynamicType ([#70120](https://github.com/pytorch/pytorch/pull/70120))
- Do not reuse mobile type parser for all unpicklers ([#71048](https://github.com/pytorch/pytorch/pull/71048))
- Migrated {TupleType, ListType} to DynamicType ([#70205](https://github.com/pytorch/pytorch/pull/70205), [#70212](https://github.com/pytorch/pytorch/pull/70212))
- Check to ensure profiler\_edge is only added when use\_kineto is on ([#67494](https://github.com/pytorch/pytorch/pull/67494))
- Removed double pragma once directive in the generated code ([#65620](https://github.com/pytorch/pytorch/pull/65620))
- Added mobile upgrader ([#67728](https://github.com/pytorch/pytorch/pull/67728), [#67729](https://github.com/pytorch/pytorch/pull/67729), [#67730](https://github.com/pytorch/pytorch/pull/67730), [#67731](https://github.com/pytorch/pytorch/pull/67731))
- Introduced multiple improvements for `operator versioning`
  - Improved compatibility APIs and minor refactor ([#68678](https://github.com/pytorch/pytorch/pull/68678), [#68677](https://github.com/pytorch/pytorch/pull/68677), [#71432](https://github.com/pytorch/pytorch/pull/71432), [#67385](https://github.com/pytorch/pytorch/pull/67385))

- Fixed some bugs in operator upgrader([#71578](https://github.com/pytorch/pytorch/pull/71578), [#70161](https://github.com/pytorch/pytorch/pull/70161), [#70225](https://github.com/pytorch/pytorch/pull/70225))
  - Used more robust way of extracting min and max versions
  - Ensured initialization thread safety

- Supported indirect method CALL in lite interpreter (bytecode)
  - Enabled `CALL` instruction in lite interpreter([#65964](https://github.com/pytorch/pytorch/pull/65964))
  - Enabled lite interpreter to correctly handle INTERFACE\_CALL instruction ([#65972](https://github.com/pytorch/pytorch/pull/65972))

## Distributed

- `torch.distributed`

- `DistributedDataParallel`

- `torch.distributed.rpc`

## TorchScript

- Fixed cases where errors were not thrown for XNNPack Ops, JIT graph executor, Cuda lowering of CUDA Tensor Lowering. This was due to uses of `TORCH_CHECK/ TORCH_INTERNAL_ASSERT` without a condition ([#71879](https://github.com/pytorch/pytorch/pull/71879), [#71767](https://github.com/pytorch/pytorch/pull/71767), [#71778](https://github.com/pytorch/pytorch/pull/71778))
- Fixed an Android SDK compilation warning when —D\_FORTIFY\_SOURCE=2 was used ([#65222](https://github.com/pytorch/pytorch/pull/65222))
- Additional warnings suppressed when compiling Caffe2 headers([#71370](https://github.com/pytorch/pytorch/pull/71370))

## Quantization

- More informative error messages from fbgemm embedding spmdm call ([#65186](https://github.com/pytorch/pytorch/pull/65186))
- Changed observer FQNs generated in prepare step ([#65420](https://github.com/pytorch/pytorch/pull/65420))
- Made FixedQParam ops work for dtypes other than quint8 ([#65484](https://github.com/pytorch/pytorch/pull/65484))
- Added op benchmark for CPU FakeQuantizePerChannel with float zero\_points ([#65241](https://github.com/pytorch/pytorch/pull/65241))
- Replaced conv\_p with convolution\_op in qnnpack ([#65783](https://github.com/pytorch/pytorch/pull/65783))
- Fixed the hypothesis test for topk ([#66057](https://github.com/pytorch/pytorch/pull/66057))
- Removed hypothesis from qtopk ([#66158](https://github.com/pytorch/pytorch/pull/66158))
- Shape propagation for quantization ([#66343](https://github.com/pytorch/pytorch/pull/66343))
- Updated observer\_fqn to not depend on node.name ([#66767](https://github.com/pytorch/pytorch/pull/66767))
- Updated qnnpack to use pytorch/cpuinfo.git repo as a third party dependency ([#67106](https://github.com/pytorch/pytorch/pull/67106))
- Added pass to duplicate dequant nodes with multi use ([#67118](https://github.com/pytorch/pytorch/pull/67118))
- Removed asymmetrical padding parameters in qnnpack ([#67102](https://github.com/pytorch/pytorch/pull/67102))
- Added out-variant for quantized::linear\_dynamic\_fp16 ([#67663](https://github.com/pytorch/pytorch/pull/67663))
- Replaced copy\_ with data\_ptr() since input Tensor’s dtype is guaranteed to be float ([#67788](https://github.com/pytorch/pytorch/pull/67788))
- Refactoring quantized op tests to combine test classes ([#68282](https://github.com/pytorch/pytorch/pull/68282))
- In q\_avgpool operator, loop over batch dimension inside operators ([#66819](https://github.com/pytorch/pytorch/pull/66819))
- Added additional string to search cpu flags for vnni detection ([#67686](https://github.com/pytorch/pytorch/pull/67686))
- Refactored handling of FixedQParams operators ([#68143](https://github.com/pytorch/pytorch/pull/68143))
- Set FakeQuant zeropoint dtype matches observer for embedding QAT ([#68390](https://github.com/pytorch/pytorch/pull/68390))
- Removed warning for quantized Tensor in ` __dir__ ` ([#69265](https://github.com/pytorch/pytorch/pull/69265))
- Moved pattern type definition to ao/quantization/utils.py ([#68769](https://github.com/pytorch/pytorch/pull/68769))
- Refactored fusion to use the new Pattern format ([#68770](https://github.com/pytorch/pytorch/pull/68770))
- Changed the type for output of convert to be torch.nn.Module ([#69959](https://github.com/pytorch/pytorch/pull/69959))
- In FX graph mode quantization, allow duplicate named\_modules during fbgemm lowering ([#70927](https://github.com/pytorch/pytorch/pull/70927))
- Added explanation of quantized comparison strategy in assert\_close ([#68911](https://github.com/pytorch/pytorch/pull/68911))
- Added quantized input tensor data type checks ([#71218](https://github.com/pytorch/pytorch/pull/71218))
- Added a guard against shapes for qnnpack qadd ([#71219](https://github.com/pytorch/pytorch/pull/71219))
- Templatized activationLimits function ([#71220](https://github.com/pytorch/pytorch/pull/71220))
- Removed unused allow list arguments from propagate\_qconfig and helper ([#71104](https://github.com/pytorch/pytorch/pull/71104))
- Supported non-partial functions in qconfig comparison ([#68067](https://github.com/pytorch/pytorch/pull/68067))

## ONNX

- Suppressed ONNX Runtime warnings in tests ([#67804](https://github.com/pytorch/pytorch/pull/67804))
- Fixed CUDA test case ([#64378](https://github.com/pytorch/pytorch/pull/64378))
- Added links to the developer documentation in the wiki ([#71609](https://github.com/pytorch/pytorch/pull/71609))

## torch.package

- Add simple backwards compatibility check for `torch.package` ([#66739](https://github.com/pytorch/pytorch/pull/66739))