PyTorch SymmetricMemory: Harnessing NVLink Programmability with Ease

XiaoSX May 25, 2025, 11:56am 3

I think they’re referring to atomicCAS based barrier update.

Topic		Replies	Views
Meta PyTorch Team 2025 H1 Roadmaps	15	4889	April 28, 2025
Memcpy based P2P communication for pipeline parallelism instead NCCL distributed	9	1345	September 4, 2024
RFC: PyTorch DistributedTensor distributed	4	6111	July 2, 2024
Rethinking PyTorch Fully Sharded Data Parallel (FSDP) from First Principles distributed	19	11075	September 17, 2024
TorchInductor Update 6: CPU backend performance update and new features in PyTorch 2.1 compiler	0	1964	September 22, 2023