Run the HIL suite unattended on self-hosted runners - #296
Draft
igoropaniuk wants to merge 8 commits into
Draft
Conversation
The abort sentinel that implements fail-fast across the serialized HIL steps is cleared at the start of flash_full, which was the suite's first step when the mechanism was introduced. The list and chipinfo steps have since been added with higher priorities, so they run before flash_full - and a failure in either sets the sentinel only for flash_full to wipe it (it is also exempted from the fail-fast check) and carry on flashing a board that just failed a sanity step. Introduce a FIRST_STEP anchor: the sentinel reset runs there and nowhere else, and no step is exempt from the fail-fast check - the first step clears the sentinel before checking, so a set sentinel always means a failure in this run. Keep FIRST_STEP in sync with the priorities in tests/meson.build. Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
The HIL suite assumes a human has already put the board into EDL mode before the suite starts, which rules out unattended execution on a CI runner. Add an optional QDL_HIL_ENTER_EDL_CMD environment hook, run in the suite's first step (after all skip gating, so runs without a configured board still skip cleanly). The environment supplies whatever mechanism the rig uses - 'adb reboot edl' on a booted board, a USB relay toggle, or a lab power controller - keeping the suite board-agnostic. The command only needs to trigger the mode switch: the flash steps already wait for the device, and the list step now polls for enumeration for up to QDL_HIL_SETTLE seconds (default 30), since a power-cycled board takes several seconds to come back as an EDL device. The suite's final reset step boots the board back into its OS, where the next run's hook can reach it again. Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
The list step passes as soon as any EDL device enumerates. On a rig with several boards attached to one host - the setup QDL_HIL_SERIAL exists for - another board sitting in EDL mode satisfies the check and masks the selected board never appearing; the suite then proceeds and fails later, in a less obvious place. When QDL_HIL_SERIAL is set, require the qdl list output to contain that serial before declaring the device present, and name the serial in the timeout diagnostic. Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
qdl's verb dispatcher scans argv for a known verb but stops at the first argument without a leading dash. With a serial configured the chipinfo step ran 'qdl -S <serial> chipinfo': the scan stops at the serial value, never sees the verb, and falls through to the flash parser, which rejects the command line with a usage message. The suite's Linux runs never caught this because they run without QDL_HIL_SERIAL. Pass the serial after the verb, where the chipinfo subcommand parses -S itself. The dispatcher's inability to step over value-taking options deserves a fix in qdl proper; this keeps the suite working against released binaries either way. Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
The hil suite exercises plain flashing; secured boards additionally support Validated Image Programming, where the device verifies the flashed data stream against signed digest tables - the feature that regressed unnoticed on Windows in v2.7. Add a dedicated hil-vip suite: list and chipinfo reuse the existing steps, and a sign-and-flash step signs a copy of the firehose programmer, has qdl generate the VIP digest tables, wraps them with mbn-tool, signs them, and flashes the build with --vip-table-path. Signing follows the signimages.sh recipe from the qcom-sec-tools repository (image ids DEVICE-PROGRAMMER and VIP, signing key index 0) and requires QDL_HIL_SECTOOLS, QDL_HIL_SECPROFILE and QDL_HIL_SECKEYS on top of the usual configuration; without them the suite skips, so rigs without signing material stay green. Only the programmer and the digest tables are signed - VIP validates the data stream, so the individual images need no signatures for this test. Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
A failing HIL step reports qdl's normal output, which is often not enough to diagnose a device-side problem after the fact - and the failure may not reproduce once the board state changes, so enabling debug logging for a rerun loses the moment. Add a QDL_HIL_DEBUG environment knob that puts --debug on every qdl invocation in both suites. Rigs set it in their board configuration; the full protocol log then lands in the meson testlog for every run, so a failure ships its own post-mortem and green runs provide a known-good log to diff against. The flag is placed before subcommand verbs, which is safe for flag-type options - only value-taking options confuse the verb dispatcher. Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
The verify step extracts the device digest with a case-insensitive grep but compares it case-sensitively against the local sha256sum output. The firehose response carries uppercase hex while sha256sum prints lowercase, so the comparison only worked as long as the extraction happened to pick a lowercase rendering of the digest from qdl's output. With QDL_HIL_DEBUG enabled the first match in the output is the device's raw uppercase digest, and the step failed on identical values - caught by the first debug-enabled CI run. Lowercase both sides before comparing. Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
Add a workflow that exercises the full hardware-in-the-loop suite on self-hosted runners labeled (self-hosted, hil, <os>) for Linux, macOS and Windows, each with a device in EDL reach (for example an Arduino Uno-Q). Board specifics stay out of the repository: each runner exports QDL_HIL_BUILD, QDL_HIL_STORAGE, QDL_HIL_ENTER_EDL_CMD and friends via the actions-runner .env file, and test_hil.sh skips cleanly where they are absent. The Windows job builds under MSYS2 like build.yml, with release: false so the runner's existing installation is reused rather than downloaded per job. Triggers are restricted to workflow_dispatch and a nightly schedule: these runners execute the checked-out tree against real hardware, so the workflow must never run for pull requests from forks. A repo-wide concurrency group serializes sweeps since the runners drive physical boards. Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Until now the HIL suite assumed a human had already put the board into
EDL mode, so it could only be run by hand. This series makes the suite
fully unattended and adds a workflow that runs it nightly on self-hosted
runners with real boards attached.
The rig drives everything itself: an environment hook switches the board
into EDL at the start of a run, the suite waits for the device to
enumerate, targets it by serial on multi-board hosts, and resets it back
into its OS at the end so the next run can reach it again. Board
specifics never enter the repository - each runner describes its boards
through env files on the host, selected via repo variables or a dispatch
input. A new hil-vip suite additionally covers Validated Image
Programming end to end on secured boards (the feature that regressed
unnoticed in v2.6), and failing runs capture qdl's debug log so they ship
their own post-mortem. A few reliability fixes shaken out by unattended
operation (fail-fast ordering, serial vs verb parsing, digest case) are
included.
The workflow triggers only on workflow_dispatch and a nightly schedule -
never on pull requests, since the runners execute the checked-out tree
against physical hardware - and a concurrency group serializes sweeps.
Merging needs repo-side setup: register the runners, set
HIL_LINUX_BOARDS, and require approval for outside collaborators'
workflow runs.