Pascal GPUs (sm_61/sm_62) still fall back to CPU with `docker model run` despite #986

[FlorentPoinsaut]

Summary

I can confirm PR #986 exists and was released in v1.2.6, but in practice docker model run still uses CPU on a Pascal GPU.

It looks like the fix scope is not the effective runtime path used by published docker/model-runner CUDA images.

Environment

• Docker Model Runner client: v1.2.6
• Docker Model Runner server: v1.2.6
• Docker Engine (Linux)
• GPU: NVIDIA GeForce GTX 1080 Ti (Pascal, sm_61)
• Driver: 580.159.03

What I observed

docker model status:

• llama.cpp Running llama.cpp 72874f559

docker model logs includes:

• installed llama-server gpuSupport=false
• ggml_cuda_init: failed to initialize CUDA: no CUDA-capable device is detected
• warning: no usable GPU found, --gpu-layers option will be ignored

Host + container GPU visibility are both OK:

• nvidia-smi works on host
• docker run --rm --gpus all nvidia/cuda:12.9.0-base-ubuntu22.04 nvidia-smi works
• docker exec docker-model-runner nvidia-smi works

Why I think #986 does not cover the effective runtime path

I checked the published runtime artifact:

• extracted /app/libggml-cuda.so from docker-model-runner
• ran cuobjdump --list-elf
• found only: sm_86, sm_89, sm_120, sm_121
• no sm_61 / sm_62 present

So even though llamacpp/native/cuda.Dockerfile in this repo contains -DCMAKE_CUDA_ARCHITECTURES=61;62;70;75;80;86;89, that does not seem to be what the final published CUDA runtime is using.

From repository inspection, release flow appears to use root Dockerfile with LLAMA_UPSTREAM_IMAGE=ghcr.io/ggml-org/llama.cpp:server-cuda13-..., which may bypass the intended Pascal-enabled build path.

Request

Please ensure Pascal support is effective for docker model run in published CUDA images (at least sm_61/sm_62), not only in a non-runtime build path.

Expected behavior

On Pascal GPUs (e.g. GTX 1080 Ti), docker model run should use CUDA (not CPU fallback).

Actual behavior

CPU fallback despite v1.2.6 and despite #986 being merged.