find_package(Torch REQUIRED) fails - 2.3.1 and nightly

I’m trying to build dlprimitives/opencl backend for pytorch for 2.3.1 or nightly but it fails in find package

I run cmake:

cmake -DCMAKE_PREFIX_PATH=$VIRTUAL_ENV/lib/python3.12/site-packages/torch/share/cmake/Torch ..

And get error:

CMake Error at /home/artik/Projects/venv/pt_3.12_nightly/lib/python3.12/site-packages/torch/share/cmake/Caffe2/public/mkl.cmake:8 (target_include_directories):
  Cannot specify include directories for imported target "caffe2::mkl".
Call Stack (most recent call first):
  /home/artik/Projects/venv/pt_3.12_nightly/lib/python3.12/site-packages/torch/share/cmake/Caffe2/Caffe2Config.cmake:106 (include)
  /home/artik/Projects/venv/pt_3.12_nightly/lib/python3.12/site-packages/torch/share/cmake/Torch/TorchConfig.cmake:68 (find_package)
  CMakeLists.txt:4 (find_package)

How can I address this issues?

Hey!
I’m not 100% sure but due to the full removal caffe2, some of these cmake and includes have moved. You might need to update the build on your end if you’re pointing to some of these explicitly?

Also I would mention recent discussions on out of tree backend that didn’t exist a long time ago: RFC-0037-Interoperability-Standard-of-3rd-Backend-Integration-Mechanism by FFFrog · Pull Request #64 · pytorch/rfcs · GitHub and torch.utils.rename_privateuse1_backend — PyTorch main documentation.
Other improvements that are WIP to automatically load these extensions and make them work without an explicit import statement. And also the new unified torch.Stream/torch.Event API to allow more device-generic code.

1 Like

OK I have just found that I used too old cmake, with cmake 3.26 it passed.

But now I have other issues - GCC is too old. It looks I’ll have to upgrade from Ubuntu 18.04 sooner or later… probably sooner that I wanted to :-)

Ok very interesting I really need to read it into details :slight_smile:

thanks…

Ok… it seems lots have changed. My current code does not work with nightly.

 File "/home/artik/Projects/venv/pt_3.12_nightly/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1165, in convert
    return t.to(
           ^^^^^
ModuleNotFoundError: No module named 'torch.ocl'

For 2.3.1 it is

RuntimeError: Please register PrivateUse1HooksInterface by `RegisterPrivateUse1HooksInterface` first.

And reading the manual a looot have changes.

Is there any digest or way to know about these updates? Maybe mailing list.

It seems the opencl backend now works only with 1.13 (I think it worked with 2.0 and 2.1)

Thanks

Hey!

Yes the “PrivateUse1HooksInterface” will allow you to control a lot more components than you were able to before!
AFAIK GitHub - Ascend/torch_npu is the most extensive end to end example of using this new API.

AFAIK GitHub - Ascend/torch_npu is the most extensive end to end example of using this new API.

Ok, thanks I will look into it.

Probably will come with some more questions. At least there is a progress :slight_smile:

Really need some mailing list/push updates or something to know when there are breaking changes in Out of Tree backend support.

1 Like

Yes I would definitely agree with that. Could you leave a comment to that effect on the RFC so that we don’t forget to setup some mechanism for this?

thanks, I added a comment to the RFC