UID2-7223 Use 1 SSM session in AWS AMI build#2575
Merged
Merged
Conversation
The latest AL2023 base image (most_recent=true) regressed SSM session stability: the Packer build hangs on a task transition until SSM's 20-min idle timeout fires, then dies with StartSession 403 (no AMI produced). Pin source_ami per region to the May 25 green build: us-east-1 (UID2) ami-0236922087fa98b6e eu-central-1 (EUID) ami-08b013271cfc23534 The most_recent filter is left commented in source.pkr.hcl for an easy revert once the SSM connection regression is resolved. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Pinning the base image to the May-25 known-good AL2023 (run 26799083499) still failed identically (Timeout waiting for privilege escalation prompt, 20-min SSM idle reap). That exonerates the base image. packer (1.15.3) and the session-manager-plugin (1.2.814.0) are also identical between the green and failing runs, so the regression is in the GitHub runner image / AWS SSM session-setup path, not the AMI. Reverting the pin to restore most_recent and drop the per-region kernel-line inconsistency it introduced. This reverts commit 3d35d185b. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ist) The runner-image regression slowed per-task SSM session setup ~3-4x; recycling a session per task is what intermittently hangs the build. Enable Ansible pipelining and SSH ControlMaster/ControlPersist so the whole playbook runs over a single persistent connection (one SSM session instead of ~49), plus ServerAlive keepalives so a slow task's session isn't reaped at SSM's 20-min idle timeout. Keeps timeout=60 as a backstop. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
swibi-ttd
changed the title
swibi-ttd
changed the title
BehnamMozafari
approved these changes
Jun 2, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes the AWS operator AMI build (
build-uid2-ami.yaml), which has been hanging/failing since ~28 May.Likely cause: A GitHub runner-image update slowed per-task SSM session setup ~3–4×. Packer recycles an SSM session per Ansible task, so it intermittently exceeds the become-prompt timeout (
Timeout waiting for privilege escalation prompt) or leaves the session idle long enough to hit SSM's 20-min idle reap →StartSession 403, no artifacts.Fix (
scripts/aws/uid2-operator-ami/ansible.cfg): run the whole playbook over one persistent SSM session instead of recycling one per task —pipelining = True+ SSHControlMaster/ControlPersist+ keepalives.timeout = 60kept as a backstop.Test run 26807032561 — both UID2 & EUID AMIs built and passed E2E,
PLAY RECAP ok=49 unreachable=0(per-task cadence dropped ~45s → ~4–5s).