Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .github/workflows/check-circular-deps.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,9 @@ jobs:
- name: Workflow trigger checkout
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0

- name: Install toolkit prerequisites
run: sudo make -C toolkit install-prereqs

- name: Determine DAILY_BUILD_ID value
run: |
echo "GitHub 'base_ref': ${{ github.base_ref }}"
Expand Down
3 changes: 3 additions & 0 deletions .github/workflows/check-package-builds.yml
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,9 @@ jobs:
- name: Checkout a stable version of the specs
uses: ./.github/actions/checkout-with-stable-pkgs

- name: Install toolkit prerequisites
run: sudo make -C toolkit install-prereqs

- name: Prepare the build environment
if: ${{ matrix.build-prep != '' }}
run: |
Expand Down
3 changes: 3 additions & 0 deletions .github/workflows/check-package-cgmanifest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,9 @@ jobs:
- name: Check out code
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0

- name: Install toolkit prerequisites
run: sudo make -C toolkit install-prereqs

# This PR runner uses an older Ubuntu with rpm version 4.17, which doesn't understand some newer macros like %bcond
- name: Define missing rpm macros
run: |
Expand Down
3 changes: 3 additions & 0 deletions .github/workflows/check-source-signatures.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,9 @@ jobs:
- name: Workflow trigger checkout
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0

- name: Install toolkit prerequisites
run: sudo make -C toolkit install-prereqs

# For consistency, we use the same major/minor version of Python that Azure Linux ships
- name: Setup Python 3.12
uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
Expand Down
3 changes: 3 additions & 0 deletions .github/workflows/check-srpm-duplicates.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,9 @@ jobs:
- name: Workflow trigger checkout
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0

- name: Install toolkit prerequisites
run: sudo make -C toolkit install-prereqs

# For consistency, we use the same major/minor version of Python that Azure Linux ships
- name: Setup Python 3.12
uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/go-test-coverage.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ on:
branches: [main, dev, 1.0*, 2.0*, 3.0*, fasttrack/*]

env:
EXPECTED_GO_VERSION: "1.24"
EXPECTED_GO_VERSION: "1.25"

permissions:
contents: read
Expand Down
10 changes: 5 additions & 5 deletions toolkit/docs/building/prerequisites-mariner.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,9 @@ This page outlines the requirements for building with the Azure Linux toolkit on

### Golang Package Requirements

Different versions of Azure Linux have been validated with the following Golang packages:
The Azure Linux toolkit requires Go 1.25 or newer:

- **Azure Linux 2.0 (CBL-Mariner)**: Validated with `msft-golang-1.24.1`
- **Azure Linux 3.0**: Validated with `golang-1.24.3`
- **Azure Linux 3.0**: Supported by the current `golang` package

## Installation Methods

Expand All @@ -21,7 +20,6 @@ The make targets automatically detect your OS version and install the appropriat
```bash
# For interactive development environments (local machines)
# Installs prerequisites but doesn't modify system configuration
# Note: On Azure Linux 2.0, this will remove golang if installed in favor of msft-golang due to the golang version requirement
sudo make -C toolkit install-prereqs

# Manually configure Docker if needed
Expand All @@ -40,14 +38,16 @@ sudo make -C toolkit install-prereqs-and-configure
```

**Recommendation**:

- Use `install-prereqs` on your local development machine
- Use `install-prereqs-and-configure` in CI/CD pipelines or when you need a complete environment setup

### Method 2: Direct Script Execution

If you prefer running the script directly, use the appropriate options for your OS version:

#### For Azure Linux 3.0:
#### For Azure Linux 3.0

```bash
# Install prerequisites with standard golang
sudo ./toolkit/docs/building/prerequisites-mariner.sh
Expand Down
75 changes: 60 additions & 15 deletions toolkit/docs/building/prerequisites-ubuntu.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,59 @@ This page outlines the requirements for building with the Azure Linux toolkit on

### Golang Package Requirements

The Azure Linux toolkit on Ubuntu has been validated with the following:
The Azure Linux toolkit requires Go 1.25 or newer.

- **Ubuntu 22.04**: Validated with `golang-1.24.13` (available as `golang-1.24-go` package)
Ubuntu 26.04 packages Go 1.25 as `golang-1.25-go`, but the older releases still in common use
(22.04, 24.04) do not package it at all. Unless your `PATH` already has Go 1.25 or newer, the
prerequisites script picks a source at runtime:

1. **The distro package**, when apt has a `golang-1.25-go` candidate (Ubuntu 26.04 and newer). Go is
installed into `/usr/lib/go-1.25`. This keeps working on hosts that can reach an apt mirror but
have no access to the public internet.
2. **The upstream toolchain**, when apt has no candidate. The pinned tarball is downloaded from
[go.dev/dl](https://go.dev/dl), verified against a SHA256 checksum, and unpacked into
`/usr/local/go`. `amd64` and `arm64` are supported; on any other architecture the script stops and
asks you to install Go yourself.

Either way `go` and `gofmt` are symlinked into `/usr/bin`, so no separate step is needed.

A Go toolchain that is already installed in either location and reports 1.25 or newer is reused
as-is — the script only ever replaces a `/usr/local/go` that is too old, and says so before it does.

If some other `go` sits earlier on `PATH` than `/usr/bin` — `/usr/local/bin/go` is the usual
culprit, since both Ubuntu's default `PATH` and `sudo`'s `secure_path` list `/usr/local/bin` first —
the script names it and warns that it will be used instead. That is only a problem if the shadowing
copy is too old — remove it if the build later reports an unsupported Go version.

The pinned version and its checksums are the `GO_VERSION`/`GO_SHA256_*` variables at the top of
`prerequisites-ubuntu.sh` and must be updated together.

To use a Go you have installed yourself instead, make sure it is on `PATH` and reports 1.25 or
newer; the script will detect it and skip the download.

#### Updating the pinned Go version

Go supports only the two most recent release series: when 1.N ships, 1.(N-2) stops receiving
security fixes. This pin therefore moves on Go's release cadence rather than Azure Linux's, and
CVE fixes are a routine reason to move it. <https://go.dev/dl> lists the supported releases.

Bumping the *patch* release (1.25.x → 1.25.y) means editing `prerequisites-ubuntu.sh` alone:

1. Set `GO_VERSION`.
2. Set `GO_SHA256_AMD64` and `GO_SHA256_ARM64` from <https://go.dev/dl> in the same change — the
checksums are version-specific, and a stale one aborts the install with a message naming the
constant that needs refreshing.

Bumping the *minor* release (1.25 → 1.26) additionally requires, all in one change:

| Location | What to change |
| --- | --- |
| `prerequisites-ubuntu.sh` | `GO_APT_PACKAGE`, `GO_APT_ROOT`, and the version regex in `go_version_ok` |
| `toolkit/tools/go.mod` | the `go` directive, which `toolkit/scripts/tools.mk` turns into the build-time minimum |
| `.github/workflows/go-test-coverage.yml` | `EXPECTED_GO_VERSION`, which both selects the CI toolchain and asserts the `go.mod` directive matches |
| `prerequisites-ubuntu.md`, `prerequisites-mariner.md` | the stated minimum version |

Leaving any of them behind either fails CI or, worse, installs a toolchain the build then rejects.

## Installation Methods

Expand All @@ -22,18 +72,14 @@ The make targets automatically install the appropriate packages:
# Installs prerequisites but doesn't modify system configuration
sudo make -C toolkit install-prereqs

# Manually create Go symlinks for proper PATH integration
sudo ln -sf /usr/lib/go-1.24/bin/go /usr/bin/go
sudo ln -sf /usr/lib/go-1.24/bin/gofmt /usr/bin/gofmt

# Manually configure Docker if needed
curl -fsSL https://get.docker.com -o get-docker.sh
sudo sh get-docker.sh
sudo usermod -aG docker $USER
# Note: You will need to log out and log back in for user changes to take effect

# the above 2 steps can alternatively be done using the following command if preferred:
# sudo ./toolkit/docs/building/prerequisites-ubuntu.sh --no-install-prereqs --fix-go-links --configure-docker
# the above step can alternatively be done using the following command if preferred:
# sudo ./toolkit/docs/building/prerequisites-ubuntu.sh --no-install-prereqs --configure-docker

----------------------

Expand All @@ -43,6 +89,7 @@ sudo make -C toolkit install-prereqs-and-configure
```

**Recommendation**:

- Use `install-prereqs` on your local development machine
- Use `install-prereqs-and-configure` in CI/CD pipelines or when you need a complete environment setup

Expand All @@ -54,25 +101,23 @@ If you prefer running the script directly, you have several options:
# Basic installation with Go
sudo ./toolkit/docs/building/prerequisites-ubuntu.sh

# Manually create Go symlinks for proper PATH integration
sudo ln -sf /usr/lib/go-1.24/bin/go /usr/bin/go
sudo ln -sf /usr/lib/go-1.24/bin/gofmt /usr/bin/gofmt

# Manually configure Docker if needed
curl -fsSL https://get.docker.com -o get-docker.sh
sudo sh get-docker.sh
sudo usermod -aG docker $USER
# Note: You will need to log out and log back in for user changes to take effect

# the above 2 steps can alternatively be done using the following command if preferred:
# sudo ./toolkit/docs/building/prerequisites-ubuntu.sh --no-install-prereqs --fix-go-links --configure-docker
# the above step can alternatively be done using the following command if preferred:
# sudo ./toolkit/docs/building/prerequisites-ubuntu.sh --no-install-prereqs --configure-docker
```

## Script Options

The `prerequisites-ubuntu.sh` script supports the following options:

- `--fix-go-links`: Creates symbolic links for Go binaries to make them available in your PATH
- `--fix-go-links`: Re-creates the `/usr/bin` symlinks for the Go binaries, pointing them at
whichever Go root is installed (`/usr/lib/go-1.<minor_version>` or `/usr/local/go`). The
prerequisites installation already does this, so it is only needed to repair the links.
- `--configure-docker`: Installs Docker and adds your user to the docker group
- `--no-install-prereqs`: Skips installation of prerequisite packages
- `--help`: Displays usage information
Expand Down
141 changes: 131 additions & 10 deletions toolkit/docs/building/prerequisites-ubuntu.sh
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,123 @@
# Copyright (c) Microsoft Corporation.
# Licensed under the MIT License.

set -e
set -eo pipefail

# Go is installed separately from the other packages because its source depends on the release.
# Ubuntu 26.04 packages Go 1.25 as golang-1.25-go; 22.04 and 24.04 do not package it at all, so
# those fall back to the upstream toolchain. The distro package is preferred so that hosts which
# can reach an apt mirror but not go.dev keep working without external egress.
#
# Go supports only the two most recent release series, so this pin has to keep moving. See
# "Updating the pinned Go version" in prerequisites-ubuntu.md before changing any of it: the
# minimum version is spelled out in four other places that must move at the same time.
# Checksums are from https://go.dev/dl and must be updated together with GO_VERSION.
GO_APT_PACKAGE=golang-1.25-go
GO_APT_ROOT=/usr/lib/go-1.25
GO_VERSION=1.25.14
GO_SHA256_AMD64=a21ae5633a269bcd7e90cf767e48225633795e99d831742cbf3397064fee7712
GO_SHA256_ARM64=9bf234ea70ffec9347fdf6b22ce4add51717d3386a38a441e8c8743fceb5eaee

# Returns zero when the given go binary meets the toolkit's minimum. Prereleases of the minimum
# series must not qualify -- go1.25rc1 predates go1.25.0 and the toolkit would reject it -- while
# prereleases of later series (go1.26rc1) are new enough.
go_version_ok() {
"$1" version 2>/dev/null | grep -qE 'go1\.(25\.[0-9]|2[6-9]|[3-9][0-9])'
}

# Echoes the root of an installed Go toolchain, preferring the distro package over the upstream
# tarball, and a toolchain that meets the minimum over one that does not -- so a stale apt root
# does not get linked over a good upstream one. Returns non-zero when neither root has a Go.
go_root() {
go_root_fallback=""
for root in "$GO_APT_ROOT" /usr/local/go; do
if [ -x "$root/bin/go" ]; then
go_version_ok "$root/bin/go" && { echo "$root"; return 0; }
[ -z "$go_root_fallback" ] && go_root_fallback="$root"
fi
done

[ -n "$go_root_fallback" ] && { echo "$go_root_fallback"; return 0; }
return 1
}

# Returns zero when a Go new enough for the toolkit is already available, whether it is first on
# PATH or sitting unlinked in one of the roots this script manages.
go_is_supported() {
go_version_ok go && return 0

go_installed_root="$(go_root)" || return 1
go_version_ok "$go_installed_root/bin/go"
}

# Installs Go from the distro when it is packaged, otherwise from the pinned upstream tarball.
install_go() {
echo "Checking apt for '$GO_APT_PACKAGE' (not packaged before Ubuntu 26.04)..."
if apt install -y "$GO_APT_PACKAGE" && [ -x "$GO_APT_ROOT/bin/go" ]; then
echo "Installed $GO_APT_PACKAGE from apt."
return
fi

echo "'$GO_APT_PACKAGE' is unavailable on this release, using the upstream toolchain..."
go_arch="$(dpkg --print-architecture)"
case "$go_arch" in
amd64) go_sha256="$GO_SHA256_AMD64"; go_sha256_var=GO_SHA256_AMD64 ;;
arm64) go_sha256="$GO_SHA256_ARM64"; go_sha256_var=GO_SHA256_ARM64 ;;
*)
echo "ERROR: no upstream Go build is pinned for architecture '$go_arch'." >&2
echo "Install Go $GO_VERSION or newer manually, then re-run with --no-install-prereqs." >&2
exit 1
;;
esac

echo "Installing Go $GO_VERSION ($go_arch) from https://go.dev/dl..."
# Deliberately not 'local': the EXIT trap below runs after this function has returned, when a
# function-scoped variable would already be out of scope and the temp dir would leak.
go_tmp_dir="$(mktemp -d)"
trap 'rm -rf "$go_tmp_dir"' EXIT
curl -fsSL -o "$go_tmp_dir/go.tar.gz" \
"https://go.dev/dl/go${GO_VERSION}.linux-${go_arch}.tar.gz"
if echo "$go_sha256 $go_tmp_dir/go.tar.gz" | sha256sum --status -c -; then
echo "Checksum OK."
else
echo "ERROR: go${GO_VERSION}.linux-${go_arch}.tar.gz does not match its pinned checksum." >&2
echo " expected: $go_sha256" >&2
echo " actual: $(sha256sum < "$go_tmp_dir/go.tar.gz" | cut -d ' ' -f 1)" >&2
echo "Either the download was corrupted or tampered with, or GO_VERSION was changed in" >&2
echo "$0 without refreshing $go_sha256_var. The expected value for a given release is" >&2
echo "published at https://go.dev/dl. Nothing has been installed." >&2
exit 1
fi

if [ -e /usr/local/go ]; then
echo "WARNING: replacing the Go installation in /usr/local/go, it is older than $GO_VERSION." >&2
fi
rm -rf /usr/local/go
tar -C /usr/local -xzf "$go_tmp_dir/go.tar.gz"
}

# Points /usr/bin/go and /usr/bin/gofmt at the installed Go root, then checks the links actually win.
link_go() {
if ! go_link_root="$(go_root)"; then
echo "No Go installation in $GO_APT_ROOT or /usr/local/go, skipping Go symlinks..."
return
fi

echo "Creating Go symlinks from $go_link_root..."
ln -vsf "$go_link_root/bin/go" /usr/bin/go
ln -vsf "$go_link_root/bin/gofmt" /usr/bin/gofmt

# Ubuntu's default PATH, and sudo's secure_path, list /usr/local/bin ahead of /usr/bin, so a
# stray go left there keeps winning. Without this warning the toolkit's own version gate fails
# much later, from a different script, with nothing pointing back to the real cause.
hash -r
go_on_path="$(command -v go 2>/dev/null || true)"
if [ -n "$go_on_path" ] && ! [ "$go_on_path" -ef /usr/bin/go ]; then
echo "WARNING: '$go_on_path' precedes /usr/bin/go on PATH and will be used instead:" >&2
echo "WARNING: $("$go_on_path" version 2>&1 | head -n 1)" >&2
echo "WARNING: remove it if the build later reports an unsupported Go version." >&2
fi
}

# Define usage function
usage() {
Expand Down Expand Up @@ -46,9 +162,6 @@ while [ $# -gt 0 ]; do
done

# Install prerequisites if not disabled
# golang version pinned for stability to avoid breaking changes. As of 11-Jun-2025 we are using golang-1.23.1 on Ubuntu 22.04 since it is the most recent release available.
# When making a breaking change to the toolkit which requires a newer golang version, update this version if needed.
# If no newer version is available, suggest moving to a newer Ubuntu LTS version
if [ "$INSTALL_PREREQS" = true ]; then
echo "Installing required packages..."
apt update
Expand All @@ -59,7 +172,6 @@ if [ "$INSTALL_PREREQS" = true ]; then
gawk \
genisoimage \
git \
golang-1.24-go \
jq \
make \
openssl \
Expand All @@ -72,15 +184,24 @@ if [ "$INSTALL_PREREQS" = true ]; then
wget \
xfsprogs \
zstd

# Install Go separately from the packages above: which source it comes from depends on the
# Ubuntu release, and a toolchain that is already good enough is reused rather than replaced.
if go_is_supported; then
echo "Found a Go toolchain that meets the minimum, skipping Go installation..."
else
install_go
fi
else
echo "Skipping installation of prerequisite packages..."
fi

# Fix go 1.24 links if requested
if [ "$FIX_GO_LINKS" = true ]; then
echo "Creating Go symlinks..."
ln -vsf /usr/lib/go-1.24/bin/go /usr/bin/go
ln -vsf /usr/lib/go-1.24/bin/gofmt /usr/bin/gofmt
# Neither Go root is on PATH -- that is the whole reason --fix-go-links exists -- so installing
# prerequisites has to refresh the links as well, or the toolkit this just prepared still cannot
# find go. --fix-go-links then means "only do the links", to repair them or to pair with
# --no-install-prereqs.
if [ "$INSTALL_PREREQS" = true ] || [ "$FIX_GO_LINKS" = true ]; then
link_go
fi

# Install and configure Docker if requested
Expand Down
Loading
Loading