Install on GH200 - Torch not compiled with CUDA enabled #532
Description
I follow standard installation process as described:
uv venv
then
uv run python examples/run_grpo_math.py
Setup:
- 1x GH200
NVCC:
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2025 NVIDIA Corporation
Built on Wed_Jan_15_19:21:50_PST_2025
Cuda compilation tools, release 12.8, V12.8.61
Build cuda_12.8.r12.8/compiler.35404655_0
Python: 3.12.11
During run I am getting:
File "/net/rkinas/nvidia/RL/venvs/nemo_rl.models.generation.vllm.VllmGenerationWorker/lib/python3.12/site-packages/torch/cuda/__init
__.py", line 363, in _lazy_init
raise AssertionError("Torch not compiled with CUDA enabled")
AssertionError: Torch not compiled with CUDA enabled
torch.cuda.is_available()
False
When i install torch in different (clean) venv using
uv pip install torch==2.7.0 torchvision torchaudio --index-url https://download.pytorch.org/whl/cu128
cuda is available