What are these `_foreach_` operators and why there is no fallback?

hardware-backends
1

I’m testing OpenCL backend with 2.1.1 and noticed many calls for _foreach_ functions that are missing like aten::_foreach_mul_.Scalar.

Now while I clearly understand utility of such operators for optimizers they add huge overload for backend developers.

Is there any way to automatically fall them back to standard non-foreach functions that already implemented?

2

Also I understand that it is better to implement fused_adam since it is way more efficient, I have such function in dlprimitives but it would still require from me implementing several optimizes…