azure-pipelines: port Linux build jobs from build-git-installers#909
Closed
azure-pipelines: port Linux build jobs from build-git-installers#909
Conversation
Port the build-dependency setup from the GitHub workflow's
create-linux-unsigned-artifacts job
(.github/workflows/build-git-installers.yml). The package list
matches one for one: build-essential for the C toolchain,
tcl tk for git-gui, gettext for i18n, asciidoc and xmlto for
documentation, libcurl4-gnutls-dev / libpcre2-dev / zlib1g-dev /
libexpat-dev for the Git build flags this pipeline will use, and
curl / ca-certificates for any in-job downloads.
The GitHub workflow runs all of this inside an ubuntu:20.04 /
ubuntu:22.04 container, both to pin the resulting .deb's glibc
ABI floor and to give apt-get a root-owned filesystem. The 1ES
pool images we run on
(GitClientPME-1ESHostedPool-{intel,arm64}-pc) silently ignore a
job-level `container:` directive, so the build executes on the
bare Ubuntu host VM as the unprivileged agent user. Run apt-get
via `sudo`, and to at least pin the .deb's glibc floor to
something an audit can read back, log the running Ubuntu version,
kernel, and effective UID at the start of the job.
Two pieces of the workflow's setup are intentionally left out:
The DEBIAN_FRONTEND=noninteractive / TZ=Etc/UTC env vars exist
only to keep `tzdata` from prompting interactively when it gets
pulled in inside the container (see 842cfa4 (fixup!
release: build unsigned Ubuntu .deb package, 2025-02-13)); the
bare 1ES image already has tzdata configured and they would have
no effect.
The Node.js workaround exists to satisfy GitHub Actions' Node-based
shim, which Azure Pipelines does not need.
Re-introducing a real container later (whether via 1ES's container
option, a custom container job, or a build inside docker invoked
from a step) is a separate question.
Assisted-by: Claude Opus 4.7
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Replace the Linux dummy build with the real package build, ported from the create-linux-unsigned-artifacts job in .github/workflows/build-git-installers.yml. The pipeline already wires $(git_version) from the prereqs stage, so the version no longer comes from a tag_version job output. $(deb_arch) is taken straight from the matrix entry, dropping the GitHub workflow's runtime dpkg-architecture round-trip; the matrix already names amd64 and arm64 explicitly. Parallelism switches from the workflow's hard-coded -j5 (a runner-specific holdover) to -j$(nproc), which is the common default and adapts to whatever the 1ES pool gives us. The shell prologue switches from `set -ex` to `set -euo pipefail` so an unbound variable or a failed step in a pipeline aborts the job rather than silently producing a broken .deb. The make recipe and DEBIAN/control body match the workflow byte for byte: same DESTDIR layout, same Make flags (USE_LIBPCRE, USE_CURL_FOR_IMAP_SEND, NO_OPENSSL, NO_CROSS_DIRECTORY_HARDLINKS, ASCIIDOC8, ASCIIDOC_NO_ROFF, ASCIIDOC='TZ=UTC asciidoc'), same prefix and gitexecdir/libexecdir/htmldir, same install targets, and the same Depends list and Description text. The only intentional content change is the package version and architecture fields, which now come from $(git_version) and $(deb_arch). The sed 's/-rc/.rc/g' substitution carried over because Git's GIT-VERSION-GEN expects rc tags spelled with a dot. Output goes under $(Build.ArtifactStagingDirectory)/app/, where the existing ESRP signing template will pick it up via its '**/*.deb' pattern. The collect step below still uses the dummy 'cp -R app/* _final/' shape; tightening that is the next commit. Assisted-by: Claude Opus 4.7 Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Replace the Linux dummy collect step with a focused move of just the signed microsoft-git_<version>_<arch>.deb into $(Build.ArtifactStagingDirectory)/_final/, which the existing templateContext.outputs.pipelineArtifact already publishes as the linux_x64 / linux_arm64 artifact. The dummy version copied everything under app/* into _final/, which worked in isolation but would have started silently uploading any intermediate files (including the pkgroot/ staging tree the build step now writes alongside the .deb). Naming the file precisely also turns "ESRP signed something else" into a missing-file error rather than a silent wrong-artifact upload. Assisted-by: Claude Opus 4.7 Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
dscho
force-pushed
the
azp-build-installers-linux
branch
from
9eaa00e to
0804fa5
Compare
Member
Author
|
Superseded by #910, which covers Linux, macOS, and Windows in one branch. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Start porting the
build-git-installersworkflow from.github/workflows/build-git-installers.ymlto the Azure Pipelines release pipeline at.azure-pipelines/release.yml, replacing thedebsigsplus Key Vault GPG key flow with ESRP signing along the way. This first slice covers the Linux jobs only; Windows and macOS still ride the placeholder steps that landed earlier onmicrosoft/azp.The series is small on purpose, four commits so each step can be reviewed in isolation:
Add a
container:clause to the Linux jobs and a matchingcontainer_imagefield to eachlinux_matrixentry. The GitHub workflow deliberately builds insideubuntu:20.04andubuntu:22.04containers (the matrix comment about needing aglibc-217Node.js build for the older Ubuntu corroborates that the choice is intentional) so that the resulting.debkeeps a known glibc ABI floor. Building bare on the 1ES pool image would silently raise that floor, which is the kind of release regression that is hard to notice in CI but obvious to a downstream user on a slightly older Ubuntu.Install the build dependencies, mirroring the package list from
create-linux-unsigned-artifacts:build-essential,tcl,tk,gettext,asciidoc,xmlto,libcurl4-gnutls-dev,libpcre2-dev,zlib1g-dev,libexpat-dev,curl,ca-certificates. The container runs as root, so nosudo. TheDEBIAN_FRONTEND=noninteractiveandTZ=Etc/UTCenv vars keeptzdatafrom blocking on its interactive timezone prompt. The Node.js shim workaround the GitHub workflow needs in order to satisfyactions/checkoutdoes not apply here.Replace the dummy build with the real
make installplusdpkg-deb --build, reusing the sameDESTDIRlayout, the same Make flags (USE_LIBPCRE,USE_CURL_FOR_IMAP_SEND,NO_OPENSSL,NO_CROSS_DIRECTORY_HARDLINKS,ASCIIDOC8,ASCIIDOC_NO_ROFF,ASCIIDOC='TZ=UTC asciidoc'), the sameprefixandgitexecdir/libexecdir/htmldir, the same install targets, and the sameDepends:line and description. The only intentional content change is the version (now$(git_version)wired fromprereqs) and the architecture (now$(deb_arch)from the matrix, dropping the workflow'sdpkg-architectureround-trip). The workflow's-j5switches to-j$(nproc), andset -extoset -euo pipefailso an unbound variable or a failed pipeline aborts the job rather than silently producing a broken.deb.Stage just
microsoft-git_<version>_<arch>.debinto$(Build.ArtifactStagingDirectory)/_final/, naming the file precisely so that "ESRP signed something else" turns into a missing-file error rather than a silent wrong-artifact upload. The dummy collect step'scp -R app/*would have started silently shipping thepkgroot/staging tree alongside the.debonce the real build landed.The
create-linux-artifactsjob from the GitHub workflow (the one that pulls a GPG key from Key Vault, imports it, monkey-patchesdebsigsto drop--openpgp, then signs the.deb) is intentionally not ported. ESRP'sLinuxSignoperation with key codeCP-453387-Pgpis already wired into the dummy job and produces an equivalent signed.debwithout the Key Vault round-trip; that is exactly the "ESRP instead of the custom sign tool based procedure" the conversion is supposed to deliver.A few items deliberately out of scope here. Post-sign verification: the original workflow ran
debsigs --verify --checkafter signing, and while ESRPLinuxSignis presumed equivalent, an actual install/runtime validation on a target Ubuntu should land before any release cutover. The release stage'sGitHubRelease@1task still referenceslinux_*/.tar.gzpatterns we do not produce, so either the asset list needs to be trimmed or a tarball job needs to land alongside the.debjob. Theubuntu:20.04/ubuntu:22.04tags resolve to Docker Hub, and if 1ES SDL precludes that, swapping to the correspondingmcr.microsoft.comtags is a one-line follow-up. And of course the Windows and macOS jobs are still on the placeholder steps frommicrosoft/azp; porting those is the next slice.Local validation: a small Node.js script (Microsoft's published
azure-pipelines-vscode/service-schema.jsonplusjs-yamlandajv) confirms the YAML parses cleanly. The schema reports a fixed set of deviations aroundextends:1ES Pipeline Templates and boolean parameter defaults that the schema does not currently model; none of them reflect real problems. Not run end-to-end on a 1ES pool yet.Marked as draft for that reason: the YAML compiles, the logic mirrors a workflow that has been running in production for a long time, but the pipeline itself has not yet been triggered against the 1ES Linux pool.