Before spamming github issues, I wanted to try my luck in here. If there is a more adequate forum for this, please give a pointer.
I am trying to compile pytorch with cuda in a conda environment, but the cmake process gets stuck because of “nvtx” libraries.
- Base OS: ubuntu 22.04
- Conda environment with python 3.12
- Cuda toolkit installed into the conda env
- git checkout: release/2.4
conda install cuda-toolkit=12.2
→ cuda headers and libs end up in here:
$CONDA_PREFIX/targets/x86_64-linux/include/cuda*.h
$CONDA_PREFIX/targets/x86_64-linux/lib/libcudart.so
$CONDA_PREFIX/nsight-compute-2023.2.2/host/target-linux-x64/nvtx/include/nvtx3/
I have set
export CUDACXX=$CONDA_PREFIX/bin/nvcc
export CUDA_BIN_PATH=$CONDA_PREFIX/bin
export CMAKE_CUDA_COMPILER=$CONDA_PREFIX/bin/nvcc
export CMAKE_CUDA_TOOLKIT_INCLUDE_DIRECTORIES=$CONDA_PREFIX/include
export CUDAHOSTCXX=$CONDA_PREFIX/bin/x86_64-conda-linux-gnu-c++
export CUDA_HOME=$CONDA_PREFIX
CMake preparations crash in here:
-- Found CUDA: /home/sampsa/anaconda3/envs/torch_nvidia/targets/x86_64-linux (found version "12.2")
-- The CUDA compiler identification is NVIDIA 12.2.140
-- Detecting CUDA compiler ABI info
-- Detecting CUDA compiler ABI info - done
-- Check for working CUDA compiler: /home/sampsa/anaconda3/envs/torch_nvidia/bin/nvcc - skipped
-- Detecting CUDA compile features
-- Detecting CUDA compile features - done
-- Found CUDAToolkit: /home/sampsa/anaconda3/envs/torch_nvidia/targets/x86_64-linux/include (found version "12.2.140")
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Check if compiler accepts -pthread
-- Check if compiler accepts -pthread - yes
-- Found Threads: TRUE
CMake Error at cmake/public/cuda.cmake:70 (message):
Failed to find nvToolsExt
Call Stack (most recent call first):
cmake/Dependencies.cmake:43 (include)
CMakeLists.txt:853 (include)
Not sure if it’s missing nvtx or nvtx3. At least the latter is here:
/home/sampsa/anaconda3/envs/torch_nvidia/nsight-compute-2023.2.2/host/target-linux-x64/nvtx/include/nvtx3
I copied - just in case - the nvtx3 directory as-is into
/home/sampsa/anaconda3/envs/torch_nvidia/targets/x86_64-linux/include/
And also the contents (files) of nvtx3 into
/home/sampsa/anaconda3/envs/torch_nvidia/targets/x86_64-linux/include/
To no avail.
FindCUdaToolkit.cmake is resposible for finding nvtx. In that line it is said that “This is a shared library only” - although version 3 seems to be header-only.
What version should I use?
This is the part that searches for nvtx. No idea how can I pass parameters to the build system as env parameters for this case.
Why do we need an automated (and obscure) cmake “finder” script for the cuda directories anyway?
Wouldn’t it just be better to explicitly define all these paths by the user? - I mean, if someone is building pytorch h@m/rself (s)he probably needs to now what (s)he’s doing and where the directories are?
In this context, a template with some default path values could be provided…