Overlapping device to host copy with GPU collectives

hardware-backends NVIDIA CUDA
6

Thanks! I reported this to NCCL team.
If you’d like, you can also open an issue on NCCL’s GitHub: GitHub - NVIDIA/nccl: Optimized primitives for collective multi-GPU communication, for easier tracking.