Conversation
meta-cla
bot
added
the
CLA Signed
|
|
||
| **NOTE:** The pre-built vLLM wheels are only compatible with CUDA 12.8, though they should work with most older CUDA versions. Alternatively, you can install the corresponding vLLM pre-built wheels directly from https://download.pytorch.org/whl/nightly/cu128, for example: `uv pip install vllm-1.0.0.dev20260219+cu130-<suffix>.whl`. Ensure the build version number (e.g., `dev20260219`) matches your PyTorch nightly installation. | ||
|
|
||
| **NOTE:** If you run into a vLLM import error, you may have to set the LD_PRELOAD environment variable to point to libcublas.so. |
There was a problem hiding this comment.
Thanks, is this a common issue? Can you be more specific what is "vLLM import error" here? Maybe not all import issue are related to can not find cublas
There was a problem hiding this comment.
The error was undefined symbol: cublasGemmEx. Yeah I can add this to the README; I am not sure how common this is, but I ran into it unfortunately :/ Lmk if you think it makes sense to include this note in the README
There was a problem hiding this comment.
what was the command you needed to use on your machine to fix that?
|
|
||
| 0. Create environment with uv: | ||
| ```bash | ||
| uv venv --python 3.12 titan-rl |
There was a problem hiding this comment.
why don't we set these in a pyproject.toml? like an [rl] extra tag. then we can just uv sync --extra rl
There was a problem hiding this comment.
torchtitan haven't extensively move to uv for now, this setup step here seems easy and we can just limit our change to RL folder for now
|
|
||
| **NOTE:** The pre-built vLLM wheels are only compatible with CUDA 12.8, though they should work with most older CUDA versions. Alternatively, you can install the corresponding vLLM pre-built wheels directly from https://download.pytorch.org/whl/nightly/cu128, for example: `uv pip install vllm-1.0.0.dev20260219+cu130-<suffix>.whl`. Ensure the build version number (e.g., `dev20260219`) matches your PyTorch nightly installation. | ||
|
|
||
| **NOTE:** If you run into a vLLM import error, you may have to set the LD_PRELOAD environment variable to point to libcublas.so. |
There was a problem hiding this comment.
what was the command you needed to use on your machine to fix that?
wwwjn
left a comment
wwwjn
left a comment
There was a problem hiding this comment.
Please address the comments before merge, thank you
|
|
||
| 0. Create environment with uv: | ||
| ```bash | ||
| uv venv --python 3.12 titan-rl |
There was a problem hiding this comment.
torchtitan haven't extensively move to uv for now, this setup step here seems easy and we can just limit our change to RL folder for now
|
|
||
| **NOTE:** The pre-built vLLM wheels are only compatible with CUDA 12.8, though they should work with most older CUDA versions. Alternatively, you can install the corresponding vLLM pre-built wheels directly from https://download.pytorch.org/whl/nightly/cu128, for example: `uv pip install vllm-1.0.0.dev20260219+cu130-<suffix>.whl`. Ensure the build version number (e.g., `dev20260219`) matches your PyTorch nightly installation. | ||
|
|
||
| **NOTE:** If you run into a `undefined symbol: cublasGemmEx` error when importing vLLM, you may have to set the LD_PRELOAD environment variable to point to libcublas.so. |
There was a problem hiding this comment.
Looks like this is very related to local cuda setup issue. Can we remove this note to make README simplify?
3 participants
LD_PRELOAD