Ship the CUDA delegate in the wheel

[shoumikhin]

The problem

The wheel now ships the runtime, the CPU kernels, the quantized kernels, the XNNPACK delegate,
the thread pool and the profiler as separate shared libraries. The CUDA delegate is the one the
whole effort was for, and it is still missing.

The delegate is built as a static archive, which a C++ application cannot link from an install.
So loading a CUDA-delegated .pte fails at run time:

Backend CudaBackend is not registered

The change

Builds the CUDA delegate and its stream helper as shared libraries, ships them, and names them as
components.

executorch/lib/libexecutorch_backend_cuda.so         the delegate
executorch/lib/libexecutorch_extension_cuda.so       the process-wide stream helper

find_package(executorch REQUIRED COMPONENTS backend_cuda)
target_link_libraries(my_app PRIVATE executorch::runtime
                                     executorch::backend_cuda)

The CUDA runtime shim already shipped, so it is unchanged.

A CUDA wheel links the CUDA runtime without bundling it, the same way the PyTorch CUDA wheels do
not, so two things make that work. The wheels carrying the runtime are declared as dependencies:

nvidia-cuda-runtime, nvidia-cublas, nvidia-curand         for CUDA 13
nvidia-cuda-runtime-cu12, nvidia-cublas-cu12, ...         for CUDA 12

written out per version because the CUDA 12 packages carry a suffix and the CUDA 13 ones do not.
And the shipped libraries gain a relative search path that reaches them:

$ORIGIN/../../nvidia/cu13/lib

Two levels up from the wheel's library directory is where those wheels install, so this finds
them wherever the environment lives. Without it the delegate resolves the runtime only through
the absolute toolkit path recorded while linking, which names the build machine.

Before and after

BEFORE                                  AFTER

a CUDA .pte from C++                    a CUDA .pte from C++
  Backend CudaBackend                     runs on the GPU
  is not registered

the delegate is a static archive        libexecutorch_backend_cuda.so

libcudart found only where the          $ORIGIN/../../nvidia/cu13/lib
build machine had it                    plus a declared dependency

A CPU wheel is unchanged: no CUDA libraries, neither component defined, and a consumer asking for
one is told while configuring rather than at link time.

The stream helper's component names the library the wheel actually ships. It was still spelled for
the target that produces it, so asking for that component failed on a CUDA wheel that does contain
the library.

The CUDA libraries also gain relative paths to each other. They are split across two directories of
the wheel and reference each other in both directions: the delegate links the shims library, and the
shims library links the stream helper. Neither hop existed, so loading either one failed with a
missing shared object. It did not come up before because the delegate was a static archive absorbed
into whatever binary used it, and only became a library that has to find its own dependencies in this
change.

The delegate also gains a relative path to the shims library it links. That library is installed in a
different directory of the wheel from the delegate, and nothing connected the two, so loading the
delegate failed with a missing shared object. It did not come up before because the delegate was a
static archive absorbed into whatever binary used it, and only became a library that has to find its
own dependencies in this change.

A CPU wheel declares no CUDA dependencies even when the machine that built it has a CUDA toolkit
installed. The dependency list falls back to the installed toolkit when the release row does not name
one, which is right for a local build where the toolkit means intent, and wrong on a release builder
that has a toolkit while building a CPU row. A CPU wheel from CI declared a CUDA runtime it never
loads, several hundred megabytes a user does not need, so the fallback now requires the build option
that actually decides whether the CUDA libraries exist.

The relative hop to the CUDA runtime is sized from each library's own depth in the package. The wheel
installs libraries at more than one depth, and a hop sized for one of them lands inside this package from
the other, where nothing is found. The library that directly links the CUDA runtime is the deeper one, so
the shorter hop reached the wrong directory. Python does not notice, because PyTorch has already loaded
the CUDA runtime into the process, which is why this needed checking against a C++ application rather
than an import.

The stream helper's component is named after its library the same way every other component is, so the
naming rule holds across the whole set rather than having one exception in a released interface.

Test plan

• the relative hop reaches the CUDA runtime from every directory the wheel installs a library into,
  resolved from the paths the build actually passes rather than from tidy relative ones. Those paths
  are absolute and a source checkout often shares the package name, so locating the package from the
  front of the path found the checkout and produced a hop that climbed out of the install directory.
• both libraries ship, each with a soname matching its file name
• exactly one shipped object defines the delegate's entry point, and one the stream helper's
• the shipped delegate loads from the installed wheel, which needs the path to the shims library.
  Confirmed by pointing the loader at that directory by hand: without it the load fails with a
  missing shared object, and with it the delegate loads.
• a CPU wheel passes every existing check, defines neither component, and ships neither library
• with CUDA enabled both libraries build, the delegate links as a shared library rather than the
  archive it was before, the wheel declares the CUDA runtime, and each library carries the
  relative path to it

Ran on Linux x86_64 and aarch64 for the CPU wheel. The CUDA build and its packaging checks ran on
an aarch64 machine with a CUDA 13 toolkit.

[shoumikhin]

Stack from ghstack (oldest at bottom):

• Build and publish CUDA wheels #21668
• -> Ship the CUDA delegate in the wheel #21645
• Ship the quantized kernels as their own library #21642
• Ship a C++ SDK in the wheel #21639
• Split the runtime out of the Python bindings extension #21610

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/21645

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

This comment was automatically generated by Dr. CI and updates every 15 minutes.