Skip to content

[BUG]: Linker raises FunctionNotFoundError instead of falling back to the driver when nvJitLink < 12.3 is installed #2408

Description

@NaderAlAwar

Is this a duplicate?

Type of Bug

Runtime Error

Component

cuda.core

Describe the bug

With a CUDA 12.0 nvJitLink in the environment (nvidia-nvjitlink-cu12==12.0.140), Linker.which_backend() — and constructing a Linker — raises

cuda.bindings._internal.utils.FunctionNotFoundError: function nvJitLinkVersion is not found

instead of warning and falling back to the driver (cuLink) backend.

This is a regression: cuda-core 0.6.0 emitted the "nvJitLink is too old (<12.3)" RuntimeWarning and fell back to the driver. The behavior broke in 0.7.0 and is still present in 1.1.0 (verified with the same cuda-bindings and nvJitLink versions in all three environments).

Root cause: nvJitLink 12.0–12.2 export only versioned symbols (__nvJitLink*_12_X); the unversioned nvJitLinkVersion export was added in 12.3. cuda-core >= 0.7.0 probes availability via cuda.pathfinder._optional_cuda_import("cuda.bindings.nvjitlink", probe_function=lambda m: m.version()), but _optional_cuda_import only forgives DynamicLibNotFoundError, so the FunctionNotFoundError raised by version() escapes instead of triggering the fallback. 0.6.0 inspected the symbol pointer (_inspect_function_pointer("__nvJitLinkVersion")) rather than calling the function, which handled this correctly.

Note that import is unaffected (import cuda.core and from cuda.core import Linker succeed); the exception fires whenever the backend decision actually runs. Also note that on 12.0–12.2 cuda.bindings.nvjitlink is unusable across the board (every unversioned symbol lookup fails, not just version()), so falling back is the only viable behavior on those versions.

How to Reproduce

Linux x86_64, Python 3.12:

python3.12 -m venv repro && . repro/bin/activate
pip install "cuda-core==1.1.0" "cuda-bindings==12.9.7" "cuda-toolkit[nvjitlink]==12.0.1"
python -c "from cuda.core import Linker; print(Linker.which_backend())"

Result:

File "cuda/bindings/cynvjitlink.pyx", line 67, in cuda.bindings.cynvjitlink.nvJitLinkVersion
File "cuda/bindings/_internal/nvjitlink.pyx", line 405, in cuda.bindings._internal.nvjitlink._nvJitLinkVersion
cuda.bindings._internal.utils.FunctionNotFoundError: function nvJitLinkVersion is not found

Verified via /proc/self/maps that the venv's 12.0 libnvJitLink.so is the only nvJitLink mapped into the process (no system library involved). Replacing cuda-core==1.1.0 with 0.6.0 in the same environment produces the expected warning + fallback; 0.7.0 reproduces the crash.

Expected behavior

Linker.which_backend() returns "driver" (with the "nvJitLink is too old (<12.3)" RuntimeWarning), and a Linker constructed in this environment uses the cuLink backend — matching cuda-core 0.6.0.

Operating System

No response

nvidia-smi output

No response

Metadata

Metadata

Labels

bugSomething isn't working

Type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions