Skip to content

Improve scripted building #542

@jenstroeger

Description

@jenstroeger

Imagine Github is down and Action workflows aren’t available, and you want to publish the next release of your package… 😳

We currently still have small blobs of code in our Action workflows that could move into the Makefile. The goal of this exercise is to be able to use the Makefile locally and be able to build and publish a package release locally. The Action workflows should then simply use these Makefile goals, too.

Here’s a list of new Makefile goals we can consider:

  • make dist-hash
    - name: Compute package hash
    if: matrix.os == env.ARTIFACT_OS && matrix.python == env.ARTIFACT_PYTHON
    id: compute-hash
    shell: bash
    run: |
    set -euo pipefail
    TARBALL_PATH=$(find dist/ -type f -name "*.tar.gz")
    WHEEL_PATH=$(find dist/ -type f -name "*.whl")
    REQUIREMENTS_PATH=$(find dist/ -type f -name "*-requirements.txt")
    SBOM_PATH=$(find dist/ -type f -name "*-sbom.json")
    HTML_DOCS_PATH=$(find dist/ -type f -name "*-docs-html.zip")
    MARKDOWN_DOCS_PATH=$(find dist/ -type f -name "*-docs-md.zip")
    BUILD_EPOCH_PATH=$(find dist/ -type f -name "*-build-epoch.txt")
    DIGEST=$(sha256sum "$TARBALL_PATH" "$WHEEL_PATH" "$REQUIREMENTS_PATH" "$SBOM_PATH" "$HTML_DOCS_PATH" "$MARKDOWN_DOCS_PATH" "$BUILD_EPOCH_PATH" | base64 -w0)
    echo "Digest of artifacts is $DIGEST."
    echo "artifacts-sha256=$DIGEST" >> "$GITHUB_OUTPUT"
  • make release
    # In some cases a user may merge commits that don't cause a version bump, which causes commitizen
    # to fail with error code 21 (NoneIncrementExit). Thus we silence that particular error to avoid
    # failing this job: https://commitizen-tools.github.io/commitizen/bump/#avoid-raising-errors
    - name: Create changelog and bump
    run: cz --no-raise 21 bump --changelog --yes
    - name: Push the release
    run: |
    git push
    git push --tags
  • make release-publish
    # Uncomment the following steps to publish to a PyPI server.
    # At the moment PyPI does not provide a mechanism to publish
    # the provenance. So, users have to download the provenance from
    # the release page of the GitHub repository to verify the artifact.
    # Install Twine without using the package's Makefile to avoid
    # installing unnecessary dependencies, which is slow.
    # - name: Set up Twine
    # run: |
    # pip install --upgrade pip wheel
    # pip install 'twine ==4.0.2'
    # Pass the username, password, and PYPI repository URL via env variables.
    # Read the password from GitHub secrets or via other trusted mechanisms.
    # Do not hardcode the password in the workflow.
    # - name: Publish to PyPI server
    # run: twine upload --verbose --skip-existing dist/*.tar.gz dist/*.whl
    # env:
    # TWINE_USERNAME=<USERNAME>
    # TWINE_PASSWORD=<PASSWORD>
    # TWINE_REPOSITORY_URL=<REPOSITORY_URL>
  • make release-gh
    - name: Create Release Notes
    run: cz changelog --dry-run "$(cz version --project)" > RELEASE_NOTES.md
    # Create the release including the artifacts and the SLSA L3 provenance.
    - name: Upload assets
    id: upload-assets
    env:
    GH_TOKEN: ${{ secrets.REPO_ACCESS_TOKEN }}
    run: |
    TAG=$(git describe --tags --abbrev=0)
    gh release create "$TAG" dist/* --title "$TAG" --notes-file RELEASE_NOTES.md
    echo "release-tag=$TAG" >> "$GITHUB_OUTPUT"
    echo "release-url=$(gh release view """$TAG""" --json url --jq .url)" >> "$GITHUB_OUTPUT"
    and
    - name: Upload provenance
    run: gh release upload ${{ needs.release.outputs.release-tag }} ${{ needs.provenance.outputs.provenance-name }}
    env:
    GH_TOKEN: ${{ secrets.REPO_ACCESS_TOKEN }}

The _release-notifications.yaml and _wiki-documentation.yaml Actions, in their entirety, could also be hoisted into the Makefile and be made part of the release.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions