Support ctypes.CDLL patching for lib function name conversion #20
+617
−4
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds automatic patching of
ctypes.CDLLto transparently translate CUDA/NCCL function names to their MUSA/MCCL equivalents when loading MUSA libraries. This enables projects like sglang that use ctypes to directly call CUDA runtime functions to work on MUSA without any code changes.Motivation
Projects like sglang use
ctypes.CDLLto load CUDA libraries and call functions likecudaMalloc,cudaIpcOpenMemHandle,ncclAllReduce, etc. On MUSA, these libraries have different names (libmusart.soinstead oflibcudart.so) and function names (musaMallocinstead ofcudaMalloc).Previously, projects needed to add explicit name translation logic (see sglang PR #17499). With this change, they just need to
import torchadaand their existing code works unchanged.Changes
Core Implementation (
src/torchada/_patch.py):_CDLLWrapperclass that wrapsctypes.CDLLinstances for MUSA libraries__getattr__to automatically translate function names:cudaXxx→musaXxx(for libmusart.so)ncclXxx→mcclXxx(for libmccl.so)cublasXxx→mublasXxx(for libmublas.so)curandXxx→murandXxx(for libmurand.so)_patch_ctypes_cdll()patch function that replacesctypes.CDLLwith a version that returns wrapped instances for MUSA librariesRuntime Utilities (
src/torchada/_runtime.py):cuda_to_musa_name()nccl_to_mccl_name()cublas_to_mublas_name()curand_to_murand_name()Tests (
tests/test_mappings.py):TestCDLLWrapperclass with real library tests (not mocks):test_libmusart_cuda_to_musa_translation- Loads actual libmusart.so, verifies cudaMalloc/cudaFree translationtest_libmccl_nccl_to_mccl_translation- Loads actual libmccl.so, verifies ncclAllReduce translationtest_libmublas_cublas_to_mublas_translation- Loads actual libmublas.sotest_libmurand_curand_to_murand_translation- Loads actual libmurand.sotest_non_musa_lib_no_translation- Verifies non-MUSA libraries aren't wrappedtest_sglang_cuda_wrapper_pattern- End-to-end test simulating sglang's exact usage pattern, actually allocates/frees GPU memoryTestRuntimeNameConversionclass for utility function testsDocumentation:
Usage
Notes
is_musa_platform() == True).Testing
All tests pass in both torch_musa 2.7.0 and 2.7.1 containers: