-
Notifications
You must be signed in to change notification settings - Fork 7
chore: reduce COPY's and wget's #814
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
rjaegers
wants to merge
24
commits into
main
Choose a base branch
from
feature/reduce-unnecessary-copy-and-wget
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Changes from all commits
Commits
Show all changes
24 commits
Select commit
Hold shift + click to select a range
41e6ff7
chore: reduce COPY's and wget's
rjaegers 8624c61
chore: fix build issue
rjaegers 0b2e86f
chore: test-drive multi-stage downloads
rjaegers ca8c04b
chore: fix warnings and build error
rjaegers 620626a
chore: more fixing
rjaegers 9188d3f
chore: move extraction of archive
rjaegers 73cfe9a
chore: fix arg reference
rjaegers 7af1004
chore: re-fix arg usage
rjaegers 5114443
chore: fix extraction
rjaegers c6fd52e
chore: squash another layer
rjaegers 5831465
chore: add checksums to gpg keys
rjaegers f389335
chore: fix line continuation
rjaegers eafb9d5
chore: fix an "it was late"
rjaegers 4a88582
chore: correct installation of certificate
rjaegers a45aba4
chore: move download of arm gcc toolchain to separate stage
rjaegers a1d0653
chore: fix gcc-arm-none-eabi install
rjaegers a448ac3
chore: hoist-up xwin as well
rjaegers 0e504ff
chore: hoist-up bats installation
rjaegers d92864e
chore: reduce number of layers
rjaegers b9daef1
Merge branch 'main' into feature/reduce-unnecessary-copy-and-wget
rjaegers f906441
Merge branch 'main' into feature/reduce-unnecessary-copy-and-wget
rjaegers 4372863
Merge branch 'main' into feature/reduce-unnecessary-copy-and-wget
rjaegers 5a8e0b4
Merge branch 'main' into feature/reduce-unnecessary-copy-and-wget
rjaegers f761eb4
Merge branch 'main' into feature/reduce-unnecessary-copy-and-wget
rjaegers File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,19 +1,74 @@ | ||
| # syntax=docker/dockerfile:1 | ||
|
|
||
| ARG BATS_VERSION=1.12.0 | ||
| ARG DOCKER_VERSION=28.2.2 | ||
| ARG XWIN_VERSION=0.6.5 | ||
|
|
||
| FROM scratch AS downloader-amd64 | ||
| ARG DOCKER_VERSION | ||
| ARG XWIN_VERSION | ||
| ADD --checksum=sha256:53dc06dc3f775282ddff93b2d0974bbe6480761018c61e7a797342a9c7f14e23 \ | ||
| https://download.docker.com/linux/static/stable/x86_64/docker-${DOCKER_VERSION}.tgz /docker.tgz | ||
| ADD --checksum=sha256:62a63b981fe391a9cbad7ef51b17e49aeaa3e7b0d029b36ca1e9c3b2a9b78823 \ | ||
| https://developer.arm.com/-/media/Files/downloads/gnu/14.2.rel1/binrel/arm-gnu-toolchain-14.2.rel1-x86_64-arm-none-eabi.tar.xz /arm-gnu-toolchain.tar.xz | ||
| ADD --checksum=sha256:9fd53950b064d067f42428a69453b927656cae68dbd7f8d3f86dcb81c80dd22d \ | ||
| https://github.com/Jake-Shadle/xwin/releases/download/${XWIN_VERSION}/xwin-${XWIN_VERSION}-x86_64-unknown-linux-musl.tar.gz /xwin.tar.gz | ||
|
|
||
| FROM scratch AS downloader-arm64 | ||
| ARG DOCKER_VERSION | ||
| ARG XWIN_VERSION | ||
| ADD --checksum=sha256:c9506628d1845c69230df3d87e2eedd7980caf8bd966f5e6a5d7a2364de42497 \ | ||
| https://download.docker.com/linux/static/stable/aarch64/docker-${DOCKER_VERSION}.tgz /docker.tgz | ||
| ADD --checksum=sha256:87330bab085dd8749d4ed0ad633674b9dc48b237b61069e3b481abd364d0a684 \ | ||
| https://developer.arm.com/-/media/Files/downloads/gnu/14.2.rel1/binrel/arm-gnu-toolchain-14.2.rel1-aarch64-arm-none-eabi.tar.xz /arm-gnu-toolchain.tar.xz | ||
| ADD --checksum=sha256:5e131007fad7c5f30d2f41090b49937fb8f16a787e5a95b4b3140e88d174dab2 \ | ||
| https://github.com/Jake-Shadle/xwin/releases/download/${XWIN_VERSION}/xwin-${XWIN_VERSION}-aarch64-unknown-linux-musl.tar.gz /xwin.tar.gz | ||
|
|
||
| FROM downloader-$TARGETARCH AS downloader | ||
|
||
| ARG BATS_VERSION | ||
| ADD --checksum=sha256:ce6eee4130298f79b0e0f09a89f93c1bc711cd68e7e3182d37c8e96c5227e2f0 \ | ||
| https://apt.llvm.org/llvm-snapshot.gpg.key /llvm.gpg.key | ||
| ADD --checksum=sha256:db2938ce5fd422f2db7a07508452772c945135d99274004c462190c323fefcf1 \ | ||
| https://dl.cloudsmith.io/public/mull-project/mull-stable/gpg.41DB35380DE6BD6F.key /mull.gpg.key | ||
| ADD --checksum=sha256:e36b020436228262731e3319ed013d84fcd7c4bd97a1b34dee33d170e9ae6bab \ | ||
| https://github.com/bats-core/bats-core/archive/refs/tags/v${BATS_VERSION}.tar.gz /bats-core.tar.gz | ||
| ADD --checksum=sha256:a122d4080a26c1da986bd0e7202b1630eb661a624915ef244f496fdd306e85fb \ | ||
| https://www.cisco.com/security/pki/certs/ciscoumbrellaroot.pem /cisco-umbrella.crt | ||
| ADD https://github.com/bats-core/bats-support#v0.3.0 / | ||
| ADD https://github.com/bats-core/bats-assert#v2.1.0 / | ||
|
|
||
| FROM ubuntu:24.04@sha256:b59d21599a2b151e23eea5f6602f4af4d7d31c4e236d22bf0b62b86d2e386b8f AS extractor | ||
|
|
||
| ARG XWIN_VERSION | ||
|
|
||
| WORKDIR / | ||
|
|
||
| # hadolint ignore=DL3008 | ||
| RUN --mount=from=downloader,target=/dl \ | ||
| --mount=type=cache,target=/var/cache/apt,sharing=locked \ | ||
| --mount=type=cache,target=/var/lib/apt,sharing=locked \ | ||
| apt-get update && apt-get install -y --no-install-recommends xz-utils \ | ||
| && tar xzf /dl/bats-core.tar.gz && mv bats-core-*/ bats-core \ | ||
| && tar xzf /dl/docker.tgz \ | ||
| && tar xzf /dl/xwin.tar.gz --strip-components=1 "xwin-${XWIN_VERSION}-$(uname -m)-unknown-linux-musl/xwin" \ | ||
| && tar xJf /dl/arm-gnu-toolchain.tar.xz --exclude='*arm-none-eabi-gdb*' --exclude='share' \ | ||
| && mv arm-gnu-toolchain-*/ gcc-arm-none-eabi | ||
|
|
||
| FROM ubuntu:24.04@sha256:b59d21599a2b151e23eea5f6602f4af4d7d31c4e236d22bf0b62b86d2e386b8f | ||
|
|
||
| ARG BATS_VERSION=1.11.0 | ||
| ARG CCACHE_VERSION=4.11 | ||
| ARG CLANG_VERSION=18 | ||
| ARG CPM_VERSION=0.40.2 | ||
| ARG INCLUDE_WHAT_YOU_USE_VERSION=0.22 | ||
| ARG XWIN_VERSION=0.6.5 | ||
|
|
||
| ARG DEBIAN_FRONTEND=noninteractive | ||
|
|
||
| HEALTHCHECK NONE | ||
|
|
||
| SHELL ["/bin/bash", "-o", "pipefail", "-c"] | ||
|
|
||
| # Set default environment options | ||
| # Set default environment options for CMake, ccache, and CPM | ||
| # and update the PATH to include the toolchain binaries | ||
| ENV CCACHE_DIR=/cache/.ccache \ | ||
| CMAKE_EXPORT_COMPILE_COMMANDS="On" \ | ||
| CMAKE_GENERATOR="Ninja" \ | ||
|
|
@@ -23,51 +78,40 @@ | |
| PYTHONPYCACHEPREFIX=/cache/.python | ||
|
|
||
| # Install the base system with all tool dependencies | ||
| # Some tools are installed via pip to get more recent versions | ||
| # hadolint ignore=DL3008 | ||
| RUN --mount=type=bind,source=.devcontainer/cpp/apt-requirements-base.json,target=/tmp/apt-requirements-base.json \ | ||
| --mount=type=bind,source=.devcontainer/cpp/apt-requirements-clang.json,target=/tmp/apt-requirements-clang.json \ | ||
| --mount=type=bind,source=.devcontainer/cpp/requirements.txt,target=/tmp/requirements.txt \ | ||
| --mount=type=cache,target=/cache,sharing=locked \ | ||
| --mount=type=cache,target=/var/cache/apt,sharing=locked \ | ||
| --mount=type=cache,target=/var/lib/apt,sharing=locked \ | ||
| --mount=type=cache,target=/var/log,sharing=locked \ | ||
| --mount=from=downloader,target=/dl \ | ||
| --mount=from=extractor,target=/src \ | ||
| apt-get update && apt-get install -y --no-install-recommends jq \ | ||
| && jq -r 'to_entries | .[] | .key + "=" + .value' /tmp/apt-requirements-base.json | \ | ||
| xargs apt-get install -y --no-install-recommends \ | ||
| # Include the Cisco Umbrella PKI Root | ||
| && wget -qO /usr/local/share/ca-certificates/Cisco_Umbrella_Root_CA.crt https://www.cisco.com/security/pki/certs/ciscoumbrellaroot.pem \ | ||
| && cp /dl/cisco-umbrella.crt /usr/local/share/ca-certificates/cisco-umbrella.crt \ | ||
| && update-ca-certificates \ | ||
| # Install some tools via pip to get more recent versions | ||
| && python3 -m pip install --break-system-packages --require-hashes --no-cache-dir --no-compile -r /tmp/requirements.txt | ||
|
|
||
| # Install clang toolchain and mull mutation testing framework | ||
| RUN --mount=type=bind,source=.devcontainer/cpp/apt-requirements-clang.json,target=/tmp/apt-requirements-clang.json \ | ||
| --mount=type=cache,target=/var/cache/apt,sharing=locked \ | ||
| --mount=type=cache,target=/var/lib/apt,sharing=locked \ | ||
| --mount=type=cache,target=/var/log,sharing=locked \ | ||
| --mount=type=cache,target=/tmp,sharing=locked,mode=1777 \ | ||
| wget -qO - https://apt.llvm.org/llvm-snapshot.gpg.key | gpg --dearmor -o /usr/share/keyrings/llvm-snapshot-keyring.gpg \ | ||
| && wget -qO - https://dl.cloudsmith.io/public/mull-project/mull-stable/gpg.41DB35380DE6BD6F.key | gpg --dearmor -o /usr/share/keyrings/mull-project-mull-stable-archive-keyring.gpg \ | ||
| && python3 -m pip install --break-system-packages --require-hashes --no-cache-dir --no-compile -r /tmp/requirements.txt \ | ||
| # Copy and install tools from the extractor stage | ||
| && bash /src/bats-core/install.sh /usr/local \ | ||
| && cp -r /dl/bats-support /usr/local/bats-support \ | ||
| && cp -r /dl/bats-assert /usr/local/bats-assert \ | ||
| && cp /src/docker/docker /usr/local/bin \ | ||
| && cp /src/xwin /usr/local/bin/xwin \ | ||
| && cp -r /src/gcc-arm-none-eabi /opt/gcc-arm-none-eabi \ | ||
| # Install clang toolchain and mull mutation testing framework | ||
| && cat /dl/llvm.gpg.key | gpg --dearmor -o /usr/share/keyrings/llvm-snapshot-keyring.gpg \ | ||
| && cat /dl/mull.gpg.key | gpg --dearmor -o /usr/share/keyrings/mull-project-mull-stable-archive-keyring.gpg \ | ||
| && UBUNTU_CODENAME=$(grep '^UBUNTU_CODENAME=' /etc/os-release | cut -d= -f2) \ | ||
| && echo "deb [signed-by=/usr/share/keyrings/llvm-snapshot-keyring.gpg] http://apt.llvm.org/${UBUNTU_CODENAME}/ llvm-toolchain-${UBUNTU_CODENAME}-${CLANG_VERSION} main" | tee /etc/apt/sources.list.d/llvm.list > /dev/null \ | ||
| && echo "deb [signed-by=/usr/share/keyrings/mull-project-mull-stable-archive-keyring.gpg] https://dl.cloudsmith.io/public/mull-project/mull-stable/deb/ubuntu ${UBUNTU_CODENAME} main" | tee /etc/apt/sources.list.d/mull-project-mull-stable.list > /dev/null \ | ||
| && echo -e 'Package: *\nPin: origin "apt.llvm.org"\nPin-Priority: 1000' > /etc/apt/preferences \ | ||
| && apt-get update \ | ||
| && jq -r 'to_entries | .[] | .key + "=" + .value' /tmp/apt-requirements-clang.json | xargs apt-get install -y --no-install-recommends | ||
|
|
||
| # Install arm-gcc toolchain | ||
| RUN mkdir /opt/gcc-arm-none-eabi \ | ||
| && wget -qO - "https://developer.arm.com/-/media/Files/downloads/gnu/14.2.rel1/binrel/arm-gnu-toolchain-14.2.rel1-$(uname -m)-arm-none-eabi.tar.xz" | tar --exclude='*arm-none-eabi-gdb*' --exclude='share' --strip-components=1 -xJC /opt/gcc-arm-none-eabi | ||
|
|
||
| # Install bats | ||
| RUN batstmp="$(mktemp -d /tmp/bats-core-${BATS_VERSION}.XXXX)" \ | ||
| && wget -qO - https://github.com/bats-core/bats-core/archive/refs/tags/v${BATS_VERSION}.tar.gz | tar xz -C "${batstmp}" \ | ||
| && bash "${batstmp}/bats-core-${BATS_VERSION}/install.sh" /usr/local \ | ||
| && rm -rf "${batstmp}" \ | ||
| && git -C /usr/local clone -b v0.3.0 https://github.com/bats-core/bats-support.git \ | ||
| && git -C /usr/local clone -b v2.1.0 https://github.com/bats-core/bats-assert.git | ||
|
|
||
| # Install xwin | ||
| RUN wget -qO - "https://github.com/Jake-Shadle/xwin/releases/download/${XWIN_VERSION}/xwin-${XWIN_VERSION}-$(uname -m)-unknown-linux-musl.tar.gz" | tar -xzv -C /usr/local/bin --strip-components=1 "xwin-${XWIN_VERSION}-$(uname -m)-unknown-linux-musl/xwin" | ||
| && jq -r 'to_entries | .[] | .key + "=" + .value' /tmp/apt-requirements-clang.json | \ | ||
| xargs apt-get install -y --no-install-recommends | ||
|
|
||
| # Compile and install additional clang tools; often necessary as binary arm64 builds are lacking, or packages are out-of-date | ||
| # Install ccache from source for a recent version | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Check failure
Code scanning / Hadolint (MegaLinter DOCKERFILE_HADOLINT)
invalid flag: --checksum Error