ci: refactor release workflows for atomic manifest updates

[adilhusain-s]

Problem

Race conditions in manifest updates: multiple architecture builds commit to git simultaneously, causing conflicts and non-deterministic results.

Current flow:

1. Each architecture builds Python
2. Each commits manifest changes directly → race conditions
3. Retry on conflict → masks problems, doesn't solve them

Solution

Decouple builds from git operations using single-writer pattern.

New flow:

1. Each architecture builds Python and generates partial manifest JSON (no git ops)
2. Upload partials as workflow artifacts
3. Single aggregation job downloads all partials and commits once atomically

Changes

reusable-release-python-tar.yml (-122 lines)

• Removed: All git operations (checkout, pull, commit, push)
• Removed: 100+ line bash asset parsing script
• Added: Call generate_partial_manifest.py (from PR feat(tooling): add partial manifest generation and application scripts #23)
• Added: Upload partial manifest as artifact
• Renamed: release-assets-and-update-manifest → release-assets-and-generate-json

release-latest-python-tag.yml & release-matching-python-tags.yml (+75 lines each)

• Added: update-manifests job:
  • Downloads all partial manifest artifacts
  • Runs apply_partial_manifests.py (from PR feat(tooling): add partial manifest generation and application scripts #23)
  • Creates/updates version manifest files
  • Single atomic commit with concurrency control
  • Idempotent: Safe to retrigger, recollects all available artifacts
• Changed: fail-fast: false - preserve successful builds if one fails

Minor Updates

• generate_tar.yml, merge-manifest.yml, python-sample.yml: Added fail-fast: false

Impact

✅ Eliminates race conditions - single commit point
✅ Safe job retrigger - failed jobs can be rerun independently
✅ Idempotent aggregation - recollects artifacts and rebuilds manifests safely
✅ Atomic updates - all-or-nothing commits
✅ Simpler code - removed ~150 lines of complex bash
✅ Testable - manifest generation tested in PR #23

Recovery Example

If s390x build fails but ppc64le succeeds:

1. ppc64le uploads partial manifest artifact
2. s390x job fails (no artifact uploaded)
3. Aggregation job downloads available artifacts (ppc64le only)
4. Creates version manifest files and commits ppc64le entries

To recover:

1. Retrigger only the failed s390x job
2. s390x now uploads its partial manifest artifact
3. Aggregation job automatically retriggered
4. Downloads all available artifacts (ppc64le + s390x)
5. Recollects and recreates version manifest files with both architectures
6. Commits atomically - no conflicts, no lost data

Key: Aggregation is independent and idempotent - safely retriggering jobs causes it to recollect all artifacts and rebuild manifests.

Dependencies

• Depends on: PR feat(tooling): add partial manifest generation and application scripts #23 (provides tooling)

Related

• Issue Release Pipeline Reliability and Manifest Correctness #15: Pipeline reliability investigation
• PR fix(infra): enhance build reliability and security scanning #22: Retry logic for reliability
• PR feat(tooling): add partial manifest generation and application scripts #23: Partial manifest tooling
• PR ci: add backfill workflow for manifest regeneration #24: Backfill workflow