datumctl compute build has multiple first-run blockers: BuildKit connectivity, registry auth, and inspect

[scotwells]

Problem

datumctl compute build — the flagship command of the compute CLI plugin from #113 — is blocked for a new user by several independent first-run obstacles that sit around an otherwise sound builder. Two of them are universal: every user, on every platform and every container runtime, hits them. Two more affect anyone whose Docker setup is not on the default socket, or whose registry credentials live in an auto-detected credential store rather than one written into ~/.docker/config.json. In each case the error names something other than the actual problem, and there is no in-product escape hatch.

The original report framed this as a macOS + current-Docker problem. That framing was too narrow. A follow-up investigation against the code showed most of these findings are runtime-independent — Docker Desktop users hit the same walls — and that the two environment-dependent ones are not caused by colima. Colima was explicitly ruled out as a cause: it serves everything the code needs, and the build succeeds end to end on colima once the endpoint is pointed at the right socket.

This remains a first-run-experience issue, not a broken feature. The builder itself is correct (see the closing section) — the work is in connectivity defaults, credential resolution, flags, and what build inspect will accept.

Environment

macOS (darwin/arm64), Docker 29.2.1 backed by colima, plugin built from #113 head (merge commit dca05e2, version string v0.0.0-pr113-dca05e2), installed as a managed datumctl plugin. Build target was a Go hello-world example built from its Dockerfile. This is the environment the findings were originally observed in, not the scope of the findings — see each finding for its actual blast radius.

Findings

1a. Universal: docker-container:// and ssh:// BuildKit addresses cannot work at all

The BuildKit connection-helper drivers are not linked into the binary. Those drivers self-register through init() side effects, and the plugin has no blank import for them — go list -deps returns only the empty connhelper registry package, and none of connhelper/dockercontainer, connhelper/ssh, connhelper/podmancontainer, or connhelper/kubepod. With nothing registered, GetConnectionHelper returns nil and gRPC dials the raw address string verbatim.

The consequence is that BUILDKIT_HOST=docker-container://… — the address docker buildx create hands you, the form essentially every buildx document suggests, and the exact form the hello-go example's own README instructs — fails with an unintelligible error:

connecting to configured BuildKit client: ... dial tcp: lookup tcp///buildx_buildkit_milobuilder0: unknown port

This is not platform- or runtime-specific. Docker Desktop users are affected identically. The path we document is a path that cannot work as shipped.

1b. Any runtime not on the default socket: BuildKit auto-detection ignores Docker contexts

connectBuildkit (internal/cmd/compute/build/buildkit.go:360-391) probes three things in order: BUILDKIT_HOST; then bkappdefaults.Address, which on darwin is unix:///var/run/buildkit/buildkitd.sock — a standalone buildkitd socket that does not exist on a macOS host; then connectDockerBuildkit, which constructs a Docker client from dockerclient.FromEnv. That constructor honors only DOCKER_HOST, and otherwise hardcodes unix:///var/run/docker.sock. It never reads ~/.docker/contexts, because docker context use is a docker/cli feature rather than a client-library one.

So a user on colima, Rancher Desktop, Podman with a docker-compatible socket, or any remote context gets "could not connect to BuildKit" on a machine where docker build works fine.

This was confirmed not to be a colima limitation. Colima's dockerd serves the /grpc endpoint the code needs: _ping returns 200, and POST /grpc returns 500 no upgrade proto in request, meaning the route exists and is simply being probed without an upgrade. Setting DOCKER_HOST to the colima socket makes the build succeed end to end with no other change. Docker Desktop normally works here only because it installs a live /var/run/docker.sock; Desktop users who turn off "Allow the default Docker socket to be used" fail exactly the same way.

Suggested fix: resolve the endpoint through the Docker context store rather than environment-only, and expose and document both --buildkit-host and DOCKER_HOST.

2. Push and build inspect ignore auto-detected credential stores, producing a bare 401

Both the push path (destination.go:95) and build inspect (inspect.go:98) resolve credentials via authn.DefaultKeychain. At the pinned go-containerregistry v0.21.6, that keychain loads ~/.docker/config.json through docker/cli's config.Load, and honors credsStore/credHelpers only when those keys are actually written in the file. It never applies the platform default store: credentials.DetectDefaultStore is applied only by config.LoadDefaultConfigFile, which the docker CLI itself calls at startup. That is precisely why docker push succeeds on a machine where datumctl returns 401.

There is a telling asymmetry inside this package: the working call is already here. buildkit.go:443 uses config.LoadDefaultConfigFile to build the BuildKit auth provider, so base-image pulls during a build authenticate correctly while the push of the resulting image does not.

Affected: any user whose config.json lacks credsStore/credHelpers while credentials live in the platform default store — osxkeychain on macOS, pass on Linux. The most common way to land in that state is removing Docker Desktop, which leaves empty auths stubs behind once its "credsStore": "desktop" entry is gone. Docker Desktop users are not affected. As an unverified inference from reading the code, a fresh docker login from a brew-installed Docker CLI should write credsStore: osxkeychain and would therefore work.

The scope is narrower than first reported, but the priority should stay high: the failure surfaces as a bare 401 UNAUTHORIZED with no hint at the cause, it silently disagrees with docker push on the same machine, and the fix is a one-line change already demonstrated a few files away.

3. Universal: build inspect cannot read the output that build --output just produced

build inspect resolves every argument as a registry reference, so neither an OCI .tar archive nor an OCI layout directory can be inspected — despite its own help text describing it as the way to look at images "already written or pushed". The failures are correspondingly confusing:

datumctl compute build inspect /tmp/hello-go-image.tar
  → GET https://index.docker.io/v2//tmp/hello-go-image.tar/manifests/latest: UNAUTHORIZED

datumctl compute build inspect ./out
  → Get "https://./v2/": dial tcp: lookup .: no such host

Verified in code, and runtime-independent. Notably the capability already exists in the same package: parseOutput (destination.go:100-110) understands both .tar archives and layout directories, while inspect.go:91 unconditionally does name.ParseReference followed by remote.Get. It simply is not wired into inspect. The IMAGE argument should accept a local archive or layout directory alongside a registry reference, so inspecting what you just built is a one-step operation.

4. Universal: there is no --buildkit-host flag

buildRequest carries an Address field (internal/cmd/compute/build/buildkit.go, around line 38) that feeds connectBuildkit, but nothing binds it to a CLI flag. The only way to steer the build at a specific BuildKit endpoint is the BUILDKIT_HOST environment variable. The error message does at least name that variable, which is how the workaround for finding 1a was discoverable at all — but a first-class flag belongs here.

Given finding 1b, DOCKER_HOST is in practice the more useful lever of the two, and it is equally undocumented — nothing in the command's help or the example README mentions it.

5. Design question: a Kraftfile's presence implicitly delegates the whole build to the unikraft CLI

In an example directory that ships a Kraftfile, datumctl compute build . does not run datumctl's own builder at all — it hands the build off to the unikraft CLI. The handoff is not unannounced: kraftfile.go:70-71 prints a notice to stderr stating that the build is entirely delegated.

The real issue is that the discovery is implicit. pipeline.go:22-26 auto-discovers a Kraftfile in the build context before any Dockerfile handling, with no flag required to trigger it and no opt-out short of deleting the file. This may well be intentional, so flagging it as a decision to make explicit rather than asserting it is a bug. As it stands, users and reviewers can believe they are exercising datumctl's build path when they are not.

6. Minor, universal: published images are not byte-reproducible

Two builds off a warm cache are byte-identical, but two cache-cold builds of identical content produce different digests (sha256:134a7a47… vs sha256:00faaced…). There is no SOURCE_DATE_EPOCH handling and no timestamp rewriting anywhere in the pipeline, and addTarEntryToErofs (buildkit.go:173-175) preserves per-file mtimes into the EROFS image. So a digest cannot be used to correlate a locally previewed image with the published one, which undercuts the "preview locally, then push" workflow the two commands together imply.

Workarounds

What unblocks a user today, until the above are addressed:

• Not on the default Docker socket: set DOCKER_HOST to the real socket path (for colima, unix://$HOME/.colima/default/docker.sock). This alone was enough to take the build from failing to succeeding end to end.
• Alternatively, run a standalone moby/buildkit container listening on TCP and point BUILDKIT_HOST at it — this sidesteps both 1a and 1b, since a tcp:// address needs no connection helper.
• For pushes that 401: point DOCKER_CONFIG at a throwaway directory whose config.json contains {"credsStore": "osxkeychain"} (or pass on Linux).

The underlying builder works

Once the obstacles above were cleared, the build and publish path worked cleanly end to end: docker.io/scotwells/datum-compute-hello-world:latest published for platform kraftcloud/x86_64 as a 5.9 MiB single layer, with build inspect --analyze reporting Compatible. Nothing here suggests the builder is wrong — the work is in making the first run reach that outcome without a detour.

Related

• feat: datumctl compute plugin — deploy and manage workloads from the CLI #113 — the PR this plugin comes from.