configure --context-size applied nondeterministically; packaged 128K default loaded instead → kernel panic on 24GB Mac (3×)

[tomaDev]

Summary

docker model configure --context-size is applied nondeterministically: the value is stored and reported correctly by docker model configure show, but llama-server sometimes loads the model with its packaged default context instead. For models packaged with a large default (e.g. ai/devstral-small-2, 128K), this silently allocates a ~21GB KV cache on load — on a 24GB Apple Silicon machine this caused three kernel panics (watchdog timeout) in one afternoon.

Environment

• Docker Desktop 4.83.0 (234302), Model Runner v1.2.6, llama.cpp b9879-metal
• macOS 26.5.2 (25F84), MacBook Pro M5, 24GB unified memory
• Models from Docker Hub ai/ catalog (Q4_K_M)

Repro

docker model configure --context-size 16384 devstral-small-2
docker model configure show devstral-small-2   # reports "context-size": 16384
# trigger a load via the Anthropic endpoint:
curl http://localhost:12434/v1/messages -H "Content-Type: application/json" \
  -d '{"model":"ai/devstral-small-2","max_tokens":8,"messages":[{"role":"user","content":"hi"}]}'
docker model logs | grep n_ctx_slot

Sometimes honored, sometimes not — within the same boot and same stored config:

# 14:42 local — honored:
srv load_model: initializing, n_slots = 4, n_ctx_slot = 16384, kv_unified = 'true'
# ~15:03 local, same boot, same stored config, next load — ignored, packaged 128K used:
srv load_model: initializing, n_slots = 4, n_ctx_slot = 131072, kv_unified = 'true'

Also observed with ai/qwen3-coder (stored 32768, loaded with packaged 4096 — harmless direction, but same root cause) and earlier with ai/devstral-small-2 (stored 32768, loaded 131072).

Impact

Devstral-small-2 at 131072 ctx ≈ 13.5GB weights + ~21GB KV. macOS panic reports show llama-server at 24.9GB resident at panic time (of 24GB RAM); system froze >90s, then:

panic(cpu 0 caller ...): watchdog timeout: no checkins from watchdogd in 92 seconds

Three such panics on 2026-07-24 (14:11, 14:19, 15:04 local). Panic files available on request. Note: capping GPU wired memory (sysctl iogpu.wired_limit_mb) does not protect — llama.cpp falls back to CPU-side buffers and the machine swap-thrashes into the watchdog timeout anyway.

Expected

1. A stored context-size is applied on every load, deterministically; or if it cannot be applied, the load fails loudly instead of falling back to the packaged default.
2. Ideally: refuse to load (or warn) when weights + KV(context) exceeds physical memory — a config hiccup should not be able to kernel-panic the host.

Related

• docker-compose config not always considered #166 (compose config not always considered — same "stored but not applied" family)
• Context size mismatch #407 (context size display mismatch)
• Request: CLI / GET endpoint for runnerConfig #440 (runnerConfig visibility)