Skip to content

e2e tests for os scaffolding#3474

Merged
itsjustriley merged 4 commits intomainfrom
e2e-scaffolding-across-oses
Mar 20, 2026
Merged

e2e tests for os scaffolding#3474
itsjustriley merged 4 commits intomainfrom
e2e-scaffolding-across-oses

Conversation

@itsjustriley
Copy link
Copy Markdown
Contributor

@itsjustriley itsjustriley commented Feb 17, 2026

WHY are these changes introduced?

Fixes developer-tools-team/issues/1044

Added macOS and Windows smoke tests to existing scaffolding tests (Linux) to ensure successful scaffolding cross-platform

What changed

  • CI matrix: Expanded from Linux-only (npm/pnpm/yarn) to include macOS (npm) and Windows (npm) smoke tests
  • template-pack.ts: Replaced system tar CLI with tar-fs stream pipeline (Windows doesn't ship tar), added shell: true for Windows .cmd wrapper resolution, isAbsolute() for cross-platform path detection, and pnpm pack timeout
  • build-check.mjs: Switched from URL.pathname to fileURLToPath() (.pathname produces invalid paths on Windows like /C:/Users/...)
  • hydrogen-react/package.json: Removed quotes from cpy glob arguments for Windows cmd.exe compatibility (cmd.exe treats single quotes as literal characters rather than string delimiters)
  • CI workflow: shell: bash on all steps, ${{ runner.temp }} instead of $RUNNER_TEMP, improved lockfile verification error messages

CI cost note

macOS runners are ~10x Linux pricing. The macOS and Windows smoke tests use npm-only to keep costs manageable. Tests run fast so the per-PR cost delta is small, but we can look at only running this on main or conditionally later if needed.

HOW to test your changes?

CI runs this test automatically on this PR

Checklist

  • I've read the Contributing Guidelines
  • I've considered possible cross-platform impacts (Mac, Linux, Windows)
  • I've added a changeset if this PR contains user-facing or noteworthy changes
  • I've added tests to cover my changes
  • I've added or updated the documentation

@shopify
Copy link
Copy Markdown
Contributor

shopify bot commented Feb 17, 2026

Oxygen deployed a preview of your e2e-scaffolding-across-oses branch. Details:

Storefront Status Preview link Deployment details Last update (UTC)
Skeleton (skeleton.hydrogen.shop) ✅ Successful (Logs) Preview deployment Inspect deployment March 20, 2026 8:11 PM

Learn more about Hydrogen's GitHub integration.

@itsjustriley itsjustriley force-pushed the e2e-scaffolding-across-oses branch from 29691f8 to 362cda1 Compare March 5, 2026 17:45
@itsjustriley itsjustriley marked this pull request as ready for review March 5, 2026 17:49
@itsjustriley itsjustriley requested a review from a team as a code owner March 5, 2026 17:49
Comment thread .github/workflows/ci.yml
@binks-code-reviewer
Copy link
Copy Markdown

binks-code-reviewer bot commented Mar 5, 2026

🤖 Code Review · #projects-dev-ai for questions
React with 👍/👎 or reply — all feedback helps improve the agent.

Failed

📋 History

✅ 1 findings → ✅ No issues → ❌ Failed

@itsjustriley itsjustriley force-pushed the e2e-scaffolding-across-oses branch from 362cda1 to b8fe3cf Compare March 5, 2026 19:34
@itsjustriley itsjustriley marked this pull request as draft March 5, 2026 19:53
@itsjustriley itsjustriley force-pushed the e2e-scaffolding-across-oses branch from 5d64210 to af038a8 Compare March 5, 2026 20:26
@itsjustriley itsjustriley force-pushed the e2e-scaffolding-across-oses branch from af038a8 to e9c7e53 Compare March 18, 2026 14:38
Extend the test_scaffolding CI job from Linux-only to a multi-OS matrix:
Linux (npm, pnpm, yarn), macOS (npm), and Windows (npm).

Several cross-platform fixes were required to make Windows pass:

- template-pack.ts: Replace system `tar` with `tar-fs` streaming
  extraction. GNU tar interprets `C:` in Windows paths as a remote host
  (`host:path` rsh syntax). The streaming approach mirrors the existing
  pattern in template-downloader.ts and uses only existing dependencies.

- template-pack.ts: Add `shell: true` on Windows for execFileSync
  calls. Windows npm-installed CLIs are .cmd wrappers that CreateProcess
  cannot execute directly — the shell must resolve them via PATH.

- template-pack.ts: Use path.isAbsolute() instead of startsWith('/')
  for tarball path detection. The latter only catches Unix absolute paths
  and misses Windows paths like `C:\Users\...`.

- build-check.mjs: Use fileURLToPath() instead of URL.pathname, which
  produces invalid Windows paths (e.g., `/D:/a/...` with leading slash).

- hydrogen-react/package.json: Remove single quotes from cpy script
  arguments. cmd.exe does not strip single quotes, so cpy-cli receives
  literal quote characters in filenames and fails.

CI workflow changes:
- Explicit `shell: bash` on all steps with bash-specific syntax
- `${{ runner.temp }}` instead of `$RUNNER_TEMP` for shell-agnostic paths
- Improved lockfile verification with descriptive error messages
- Removed stale `--routes` flag (routes are now always scaffolded)
- Timeout increased 15m → 20m for slower macOS runners
- Concurrency group includes matrix.os to prevent cross-OS cancellation
Add a 60s timeout to the execFileSync('pnpm pack') call in
template-pack.ts. Without this, a hung pnpm process blocks a
developer's machine indefinitely — CI has its own 20min timeout but
local runs had no safety net.

Also add WINDOWS_SHELL_OPTS to the getCatalogVersion test helper in
template-pack.test.ts. The production code already handles Windows
shell resolution via this constant, but the test helper was missing it,
creating a latent failure if unit tests ever run on Windows.
@itsjustriley itsjustriley force-pushed the e2e-scaffolding-across-oses branch from 7bd0186 to 1e02160 Compare March 18, 2026 18:40
…, portable env

- Add filter to tarExtract so only package/package.json is extracted
  from the tarball, matching the intent of the old tar -xOf approach
- Export WINDOWS_SHELL_OPTS from template-pack.ts and import in the
  test file to avoid duplicating the platform detection logic
- Use step-level env: instead of inline VAR=value bash syntax for
  SHOPIFY_HYDROGEN_FLAG_LOCKFILE_CHECK, making it shell-agnostic
@itsjustriley itsjustriley marked this pull request as ready for review March 18, 2026 19:44
@itsjustriley
Copy link
Copy Markdown
Contributor Author

CI failures unrelated to PR

@github-actions
Copy link
Copy Markdown
Contributor

We detected some changes in packages/*/package.json or packages/*/src, and there are no updates in the .changeset.
If the changes are user-facing and should cause a version bump, run pnpm run changeset add to track your changes and include them in the next release CHANGELOG.
If you are making simple updates to examples or documentation, you do not need to add a changeset.

Comment thread packages/cli/src/lib/template-pack.ts Outdated
import {pipeline} from 'node:stream/promises';
import {tmpdir} from 'node:os';
import gunzipMaybe from 'gunzip-maybe';
import {extract as tarExtract} from 'tar-fs';
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

non-blocking: The PR description says "Windows doesn't ship tar". However, modern Window does ship tar.exe, and GitHub's windows-latest (Windows Server 2022) includes it.

The motivation for using tar-fs is valid though by avoiding cross-platform flag compatibility issues (BSD tar vs GNU tar) and provides a Node-native solution using deps already in this package. Worth correcting the PR description to reflect intent.

Comment thread .github/workflows/ci.yml
--package-manager ${{ matrix.pm }} \
--no-shortcut \
--routes \
--markets none \
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note: The --routes flag was quietly removed here. It was removed from the CLI in #3448 (routes are now auto-generated during init). Just noting for context.

The tar-fs `filter` callback received paths that never matched the
`PACKAGE_JSON_ENTRY` constant due to path normalization differences,
causing the filter to silently extract everything regardless. Rather
than fixing the path comparison, remove the filter entirely — the temp
directory is created with `mkdtemp`, cleaned up in `finally` with
`rm({recursive: true})`, and a `pnpm pack` tarball for a template is
small enough that filtering has negligible benefit.

Addresses blocking review comment on PR #3474.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@itsjustriley itsjustriley marked this pull request as draft March 20, 2026 20:09
@itsjustriley itsjustriley requested a review from andguy95 March 20, 2026 20:14
@itsjustriley itsjustriley marked this pull request as ready for review March 20, 2026 20:14
Copy link
Copy Markdown
Collaborator

@andguy95 andguy95 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm!

@itsjustriley itsjustriley merged commit 26fda67 into main Mar 20, 2026
22 of 23 checks passed
@itsjustriley itsjustriley deleted the e2e-scaffolding-across-oses branch March 20, 2026 20:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants