We want to follow the following rule for any new API being added (and are going to migrate existing API to follow this as well):
- Any API that related to device property should be done as a Device-agnostic API (all devices if possible, else accelerator) unless there is a clear reason on why it does not apply to other devices.
- Device-agnostic APIs that apply to all devices should be placed under the
torchnamespace and accept adevice_typeoptional argument (with default value being the current accelerator or cpu). - Device-agnostic APIs for accelerators style device (torch — PyTorch main documentation) should be placed under the
torch.accnamespace and not accept adevice_typeargument (they are always applied on the current accelerator). - Device-specific APIs should be placed under device-specific submodules (torch.cuda, torch.xpu, torch.mps, etc).
Please comment here if there is any concern/feedback about this new rule.
For more details on the discussions that lead to this proposal, see [RFC] A device-agnostic Python runtime API design for stream-based accelerators · Issue #128403 · pytorch/pytorch · GitHub , partial list of related issues in both core ([RFC] A device-agnostic Python device memory related API design for stream-based accelerators · Issue #134978 · pytorch/pytorch · GitHub, etc) and ao (Enable Intel GPU by dbyoung18 · Pull Request #753 · pytorch/ao · GitHub) as well as early investigative implementation (Refine the logic of device construction when only device index is given by guangyey · Pull Request #129119 · pytorch/pytorch · GitHub, Stream/Event API, Update pin memory related APIs to not pass 'device' argument by wizzniu · Pull Request #131858 · pytorch/pytorch · GitHub, etc).
The credit for driving this discussion goes to @guangyey and other people involved on the issues listed above.
