E2E: trial for more workers + shards

[phyllis-sy-wu]

WHY are these changes introduced?

Evaluate whether combining more workers (10) with sharding (2 CI machines) gives the best E2E runtime, since each shard gets its own 4 vCPU runner — avoiding the contention that hurt 10 workers on a single machine.

CI runner specs: ubuntu-latest = 4 vCPUs, 16 GB RAM (GitHub docs).

Decision: ship this PR

Cost is not a concern — GitHub Actions standard runners are free for public repositories, and Shopify/cli is public. The "2x CI machine cost" con listed below does not apply.

CPU contention is the same as baseline — with 2 shards, each machine runs ~5 workers on 4 vCPUs. That's identical to the current 5 workers / 1 shard setup. The contention concern only applied to 10 workers / 1 shard (#7343).

Only real tradeoffs:

• 2x setup overhead (build, Playwright install, OAuth auth duplicated per shard)
• Split artifacts — two Playwright reports instead of one
• 2x concurrent API pressure on the same org (up to 10 tests hitting Shopify APIs simultaneously instead of 5) — could increase transient failures, but flakiness observed so far appears to be inherent to the UI, not load-related

~4 min wall-clock savings (34% reduction) justifies these tradeoffs, especially if E2E becomes a required check. This is also a one-line config change to revert if needed.

(Slack discussion)

WHAT is this pull request doing?

• Bumps workers to 10 in playwright.config.ts
• Adds a matrix strategy with shard: ['1/2', '2/2'] to the E2E CI job
• Each shard uploads artifacts with unique names to avoid collisions

Benchmark Results:
(under stable conditions, very few teardown failures or flaky retries)

	#7309	#7343	#7349	#7350
Workers	5	10	5	10
Shards	1	1	2	2
CI runtime	11m 43s	8m 49s	8m 59s (shard 1/2) 7m 41s (shard 2/2)	7m 44s (shard 1/2) 5m 48s (shard 2/2)

10 workers + 2 shards vs 5 workers + 1 shard (baseline #7309):

Pros:

• ~4 min faster wall clock (7m 44s vs 11m 43s, 34% reduction), or even faster without teardown failure.
• Sharding distributes load across machines, reducing per-runner CPU/memory pressure

Cons:

• 2x CI machine cost (two runners instead of one) — not applicable, free for public repos
• 2x setup overhead (build, playwright install, OAuth auth — all duplicated per shard)
• Split artifacts — two separate Playwright reports to check instead of one
• 10 workers on 4 vCPU is still tight per machine — same ratio as baseline with 2 shards (5 workers per 4 vCPU)

Note: Total number of stores/apps created is the same regardless of workers/shards — it's determined by the number of tests. More workers/shards only increases how many run concurrently, not the total count.

How to test your changes?

Compare CI run times across all four trial PRs.

Checklist

• I've considered possible cross-platform impacts (Mac, Linux, Windows)
• I've considered possible documentation changes
• I've considered analytics changes to measure impact
• The change is user-facing, so I've added a changelog entry with pnpm changeset add

[phyllis-sy-wu]

• E2E: trial for more workers + shards #7350 👈 (View in Graphite)
• E2E: parallel workers #7309 : 2 other dependent PRs (#7343 , #7349 )
• E2E: per-test store isolation with automatic teardown #7279
• E2E: temporary time limit bump #7282
• E2E: improve error context in assertions #7281
• E2E: one-time global auth before test start #7272
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.

[github-actions]

Differences in type declarations

We detected differences in the type declarations generated by Typescript for this branch compared to the baseline ('main' branch). Please, review them to ensure they are backward-compatible. Here are some important things to keep in mind:

• Some seemingly private modules might be re-exported through public modules.
• If the branch is behind main you might see odd diffs, rebase main into this branch.

New type declarations

We found no new type declarations in this PR

Existing type declarations

packages/cli-kit/dist/public/node/upgrade.d.ts

@@ -1,5 +1,3 @@
-import { getAutoUpgradeEnabled, setAutoUpgradeEnabled } from '../../private/node/conf-store.js';
-export { getAutoUpgradeEnabled, setAutoUpgradeEnabled };
 /**
  * Utility function for generating an install command for the user to run
  * to install an updated version of Shopify CLI.