Hi everyone,
This is a reminder to feature owners to file a release-highlight issue for every feature you want tracked in the release.
IMPORTANT: Any feature not going through this process will NOT be mentioned in the release blog and other comms.
How to submit a feature
Please either:
- Open a new issue on pytorch/pytorch using the New Feature for Release template, or
- Tag an existing RFC / tracking issue with the release-feature-request label.
For each feature, include:
- What will be available in release 2.13 (API surface, stability tag: Stable / Beta / Prototype)
- Link(s) to relevant tutorials (new or updated)
- A short blog-post write-up (2-3 paragraphs, usable as-is in the release blog)
- Any platform or backend caveats (CUDA / ROCm / XPU / MPS / CPU-Arm)
The running list of tracked features lives here: Issues · pytorch/pytorch · GitHub
2.13 Features Identified by AI
The following major features were identified by an AI-assisted scan of the release/2.12 → release/2.13 commit diff. This list is a starting point, not authoritative — descriptions and stability tags have not been reviewed by the feature owners.
Note: If your team is responsible for one of the features below, please submit a “New Feature for Release” issue so we can track the official description, tutorial, and blog write-up. Corrections and missing features are equally welcome — reply on this thread or open an issue.
Prototype Features
[Prototype] FlexAttention on Apple Silicon (MPS) — MPS
FlexAttention is now available on Metal/MPS, with Metal kernels for both the sparse prefill and decode paths (including GQA and captured buffers). On long, sparse attention patterns the speedups over SDPA are large — e.g. on a 1×8×32768×64 shape with a 256-element sliding window (0.8% density), FlexAttention runs in ~35 ms vs ~431 ms for SDPA (~12.3×); a smaller 8192-length / 64-window case is ~4.15×. Dense patterns favor SDPA, as expected.
See #182552, #186215, #181575.
[Prototype] Python 3.15 binary support (torch) — Release Engineering
PyTorch binaries (wheels) now support Python 3.15, including the free-threaded 3.15t build, added to the Linux manywheel nightly matrix alongside advertised wheel metadata.
See #184600, #186017, #186244.
[Prototype] nn.LinearCrossEntropyLoss — NN / Frontend
A new fused linear + cross-entropy loss module (and linear_cross_entropy functional) computes logits and the loss together with chunking, avoiding materialization of the full logits tensor — useful for large-vocabulary models where the logits matrix dominates memory.
See #172446, #172286, #185852.
[Prototype] FSDP2 separate reduce-scatter group — Distributed
FSDPModule.set_separate_reduce_scatter_group(enable=True) gives reduce-scatter its own NCCL communicator so it can progress concurrently with all-gather instead of serializing on a single communicator, enabling AG/RS overlap for training throughput.
See #186335.
[Prototype] Load safetensors directly with torch.load — NN / Frontend
torch.load(“foo.safetensors”) now works, returning the tensors serialized in a safetensors file without a separate loader.
See #170592.
[Prototype] torch.compiler.set_default_backend — Dynamo / torch.compile
Override the global default torch.compile backend (mirroring torch.set_default_dtype / torch.set_default_device), so out-of-tree backend authors no longer need to pass backend= at every call site. Explicit backend= arguments still take precedence.
See #178944.
[Prototype] XPU device telemetry APIs — XPU
New query APIs for Intel GPUs expose runtime device state: torch.xpu.device_memory_used, plus utilization, power draw, clock rate, and temperature, alongside device-wide synchronization and additional device properties (last-level cache size, integrated-GPU detection).
See #183431, #183429, #183428, #183427.
[Prototype] CuTeDSL “Native DSL” backend for Inductor — Inductor
Inductor gains a CuTeDSL-based backend (with Quack-derived GEMM and RMSNorm kernel overrides). Kernel compilation moves from the thread pool to a subprocess pool (matching the Triton compilation path), since module loading is pure-Python work blocked by the GIL.
See #181267, #182108, #186310.
[Prototype] TorchComms backend — Distributed
A new TorchComms communications backend is integrated into the PyTorch Distributed CI and device-mesh paths, providing an alternative c10d backend.
See #181662, #178533, #182057. TODO(release-manager): confirm public-facing status / naming for TorchComms.
[Prototype] torch::stable::Generator — libtorch / C++ ABI
The stable C++ ABI now lets kernels retrieve an at::Generator in an ABI-stable way (previously always passed as null), with memory management mirroring the stable Tensor handling — extending torch::stable toward RNG in custom kernels.
[Prototype] Armv9-A target support for torch.compile — CPU / Arm
torch.compile on AArch64 now recognizes Armv9-A CPUs (e.g. Neoverse V2), propagating the correct target triple and feature set (128-bit and 256-bit SVE) through Inductor codegen, with no behavior change for x86 or other architectures.
See #184555.
[Prototype] torch.autograd.graph.region_activation_memory_budget — Autograd
A context manager that overrides the activation-memory budget for a region of a compiled forward, letting models dial recomputation aggressiveness per graph (recompute cheap blocks, preserve expensive ones) instead of relying on a single global knob. The AOT min-cut partitioner reads the per-region budget back from FX node metadata.
See #185979.
[Prototype] Experimental CUPTI monitor profiler — Profiler
A hidden, experimental profiler backend uses the CUPTI monitor for GPU collection (off the GIL) while reusing the existing CPU profiler path, producing merged Chrome traces with GPU-side user annotations recreated for record_function scopes.
[Prototype] CUDAGraph.get_graph_data() — CUDA
Extracts node topology, types, kernel names, and dependency edges from a captured CUDA graph, with ids remapped to match CUPTI profiler output — so graph structure can be correlated with profiled kernel launches.
See #183165.
Performance Improvements
Deterministic backward for the FlexAttention flash backend — Inductor
The FlexAttention flash backend gains a deterministic backward path (compute_dq_write_order), with measured end-to-end overhead on create_block_mask falling to well under 1% at longer sequence lengths (e.g. +0.2% at S=32768).
See #174813.
Combo-kernel autotune grouping on by default — Inductor
combo_kernel_autotune_grouping is now enabled by default, improving autotuning of fused/combined kernels out of the box.
See #182567.
Large MPS op migration off MPSGraph to native Metal — MPS
A broad set of operations moved from MPSGraph to hand-written Metal kernels — copy/cast, uniform/normal/randint, comparisons, reductions (sum/mean), cumsum/cumprod, sort (multi-block and stable), embedding backward, and scatter/gather with bounds checking — reducing dispatch overhead on Apple Silicon.
See #184740, #185609, #185119.
ROCm: AOTriton 0.12b, Origami GEMM selection, native HIP CMake — ROCm
ROCm bumps the AOTriton attention backend to 0.12b (head_dim ≠ head_dim_v, use_deterministic_algorithms, gfx1100/gfx1151 promoted out of experimental, partial FAv3 on gfx950). The Origami tool provides fast, analytical, deterministic GEMM-configuration selection to cut autotuning latency, and the build adopts CMake’s native HIP language support (enable_language(HIP)).
See #184288, #172512, #180485.
SDPA acceleration on Arm CPUs — CPU / Arm
OpenBLAS is updated to v0.3.33, adding/optimizing SBGEMM/BGEMM on SVE128/SVE256 machines (with Neoverse-V3 detection) and accelerating SDPA on Arm CPUs.
See #177012.
Deprecations and Backwards-Incompatible Changes
-
Named tensors removed. The deprecated named-tensor feature (Tensor.names and associated APIs) has been hard-removed to cut overhead and code bloat. See #173895.
-
Distributed collectives renamed to a _single scheme. all_gather_into_tensor → all_gather_single and reduce_scatter_tensor → reduce_scatter_single, aligning with TorchComms. The old names remain as thin wrappers marked deprecated via FutureWarning. See #186123.
-
Bazel build removed. The Bazel build was never broadly adopted and depended on an antiquated Bazel 6; it has been removed. See #180883.
Non-Feature Updates
-
Python support: CPython 3.13t dropped from the Linux binary matrix. (Python 3.15 binary support is listed above as a Prototype feature.) See #182951.
-
CUDA: CUDA 13.0 remains the default build; small wheels are now always built for CUDA Linux and the CUDA 12.8/12.9 builds were removed; ptxas is no longer bundled in the cu13 binary. See #180612, #174716.
-
Triton: pin advanced to 3.7.1. See #186792.
-
oneDNN: submodule upgraded to v3.12. See #181222.
2.13 Release Timeline
- M1: Release Announcement (20/5/26)
- M2: All PRs landed in PyTorch repo / Feature Submission Closed (5/6/26)
- M3: Release branch cut, create RC1 for PyTorch and Torchvision (week of 8/6/26)
- M4: Release branch finalized, Announce final launch date, Feature classifications published (week of 22/6/26) - Final RC is produced.
- M4.1: Tutorial drafts submission deadline (30/6/26)
- M5: External-Facing Content Finalized (1/7/26)
- M6: Release Day (8/7/26)
How to help
- Feature owners: file a release-highlight issue (template above) with your blog blurb + tutorial links.
- Tutorial authors: PRs to pytorch/tutorials with the 2.13-release label are tracked against M4.1.
- Everyone else: please review the feature list above and reply in this thread if something major is missing — particularly for MPS, Quantization, and CPU/Arm, where the diff-based scan caught few candidates.
If you have questions, reach out to us under this post.
Cheers, Team PyTorch