Skip to content

Let callers choose the image platform in ContainerManager.create - #842

Open
rcfa wants to merge 1 commit into
apple:mainfrom
rcfa:thread-image-platform
Open

Let callers choose the image platform in ContainerManager.create#842
rcfa wants to merge 1 commit into
apple:mainfrom
rcfa:thread-image-platform

Conversation

@rcfa

@rcfa rcfa commented Aug 18, 2026

Copy link
Copy Markdown

Summary

ContainerManager.create resolves images against .current, so a container can only be created
from an image variant matching the host architecture. This adds an optional platform parameter
(defaulting to .current) so callers can select a different variant.

The problem

Two sites on the create path pin the platform:

// Sources/Containerization/ContainerManager.swift
let imageConfig = try await image.config(for: .current).config           // :295
return try await unpacker.unpack(image, for: .current, at: destination)  // :346

Creating a container from an amd64 image on an arm64 host fails with:

platform linux/arm64

Two things make this hard to diagnose from the outside:

  1. The error names the platform that was requested, not the one the image has, so it looks
    like the image is mistagged or corrupt.
  2. It happens with rosetta: true on the ContainerManager as well — the failure is at config
    resolution
    , before the runtime reaches the point where translation applies. So the flag that
    exists to make this case work appears to do nothing, with no indication why.

The change

Image.config(for:) and EXT4Unpacker.unpack(_:for:at:progress:) already accept a platform; only
create pins it. This threads platform: Platform = .current through the three create overloads
and the private unpack.

The default preserves current behaviour exactly — the change is additive, and no existing caller
needs to change.

Why it matters in practice

We run SWE-bench's per-instance task images on Apple silicon. That corpus is mixed: of the 500
SWE-bench Verified instances, 281 publish arm64 images and 219 publish amd64 only. A host-pinned
platform therefore makes 44% of the benchmark unrunnable — and evaluating only the arm64 remainder
would produce a score over a non-random subset (architecture correlates with repository and era),
which would not be a valid benchmark result.

Verification

Tested on an arm64 host against both variants, running the same command shape in each:

image result
astropy instance (arm64) conda env activates; import astropy5.1.dev623+gd16bfe05a7
pytest instance (amd64, Rosetta) uname -mx86_64; import _pytest3.9.20

The second row is the case that could not be expressed before this change.

`create` resolves an image against `.current` in two places — the image config
and the rootfs unpack:

    let imageConfig = try await image.config(for: .current).config          // :295
    return try await unpacker.unpack(image, for: .current, at: destination) // :346

So a container can only ever be created from an image variant matching the host
architecture. On an arm64 host, creating a container from an amd64 image fails
with `platform linux/arm64` — naming the platform that was requested rather than
the one the image actually has, which reads as a corrupt or mistagged image.

This is reachable even with Rosetta enabled on the `ContainerManager`: the
failure happens at config resolution, before the runtime would apply any
translation, so `rosetta: true` does not help and gives no hint as to why.

Both underlying APIs already take a platform (`Image.config(for:)` and
`EXT4Unpacker.unpack(_:for:at:progress:)`); only the `create` path pins it. This
adds `platform: Platform = .current` to the three `create` overloads and the
private `unpack`, passing it through. The default preserves existing behaviour,
so the change is additive and no caller needs updating.

Motivation: running SWE-bench's task images on Apple silicon. That corpus is
mixed — of the 500 SWE-bench Verified instances, 281 publish arm64 images and
219 publish amd64 only — so a host-pinned platform makes 44% of the benchmark
unrunnable, and evaluating only the remainder would report a score over a
non-random subset.

Verified on an arm64 host with both variants, same command shape:

  * arm64 image  — conda env activates, `import astropy` succeeds
  * amd64 image  — `uname -m` reports x86_64 under Rosetta, `import _pytest`
                   succeeds

The amd64 case is the one that could not be expressed before.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant