Skip to content

feat: bring harness flows into latest version of cli#1598

Draft
avi-alpert wants to merge 4 commits into
aws:mainfrom
avi-alpert:aalpert/preview-cutover
Draft

feat: bring harness flows into latest version of cli#1598
avi-alpert wants to merge 4 commits into
aws:mainfrom
avi-alpert:aalpert/preview-cutover

Conversation

@avi-alpert

@avi-alpert avi-alpert commented Jun 19, 2026

Copy link
Copy Markdown
Contributor

Summary

The __PREVIEW__ / isPreviewEnabled() build-time flag gated harness and related flows (export, add/remove tool, add/remove skill) to preview-only builds. This PR cuts over so those flows are the only path: every preview-enabled branch is inlined as always-on, the GA-only branches are deleted, and the build-time machinery is removed.

Result: harness is now available in the standard (@latest) release — no preview build required.

Closes #1569

What changed

  • Remove isPreviewEnabled() + the __PREVIEW__ esbuild define, the BUILD_PREVIEW wiring in index.ts, the build:preview npm script, and the AGENTCORE_PREVIEW CDK passthrough (wrapper.ts + vended assets/cdk/bin/cdk.ts).
  • Inline all preview-gated branches across commands and TUI as always-on; delete the now-dead GA-only branches (including the old create-prompt flow).
  • Dedupe scripts/bundle.mjs dual-tarball logic — GA and preview builds are now byte-identical.
  • Fix the CI workflows that consumed the removed -preview tarball: e2e-tests.yml, prerelease-tarball.yml, canary.yml.
  • Update README.md: harness section no longer says it requires @aws/agentcore@preview.
  • Delete preview-flag.test.ts; update tests that mocked the flag.

Out of scope / notes

  • Does not touch isGatedFeaturesEnabled() / ENABLE_GATED_FEATURES — that is a separate runtime flag and is unrelated.
  • Preview publishing continues. We keep publishing the binary to the @preview npm tag so existing 1.0.0-preview.XX customers don't have to switch over
  • e2e-tests-full.yml still has a [preview, ga] matrix (now running identical builds) — left as-is for a follow-up release-pipeline simplification.

Testing

  • Unit suite, typecheck, lint, and asset snapshots all pass.
  • Harness e2e tests (formerly preview-gated) now run unconditionally in the single build.

@avi-alpert avi-alpert requested a review from a team June 19, 2026 18:16
@github-actions github-actions Bot added the size/xl PR size: XL label Jun 19, 2026
@github-actions github-actions Bot added the agentcore-harness-reviewing AgentCore Harness review in progress label Jun 19, 2026
@agentcore-devx-automation agentcore-devx-automation Bot added the claude-security-reviewing Claude Code /security-review in progress label Jun 19, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Package Tarball

aws-agentcore-0.20.2.tgz

How to install

gh release download pr-1598-tarball --repo aws/agentcore-cli --pattern "*.tgz" --dir /tmp/pr-tarball
npm install -g /tmp/pr-tarball/aws-agentcore-0.20.2.tgz

@agentcore-devx-automation

Copy link
Copy Markdown
Contributor

Claude Security Review: no high-confidence findings. (run)

@agentcore-devx-automation agentcore-devx-automation Bot removed the claude-security-reviewing Claude Code /security-review in progress label Jun 19, 2026
@avi-alpert avi-alpert marked this pull request as draft June 19, 2026 18:22

@agentcore-cli-automation agentcore-cli-automation left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cutover looks correct and the production code paths are clean. Two issues that need to be addressed before merging, plus one minor follow-up:

1. computeInvokeAttrs still has a dead preview parameter

src/cli/commands/invoke/utils.ts still gates isHarnessInvoke() on options.preview:

const isHarness = options.preview && isHarnessInvoke(options);

Both call sites this PR touches now hardcode preview: true (src/cli/commands/invoke/command.tsx:384, src/cli/tui/screens/invoke/useInvokeFlow.ts:179), so the parameter no longer carries any signal. It should be removed entirely (function param + both call sites + the preview: false test cases in src/cli/commands/invoke/__tests__/utils.test.ts). Otherwise the code reads as if there is still a preview gate when there isn't, and there's a real risk of a future caller accidentally passing preview: false and silently misclassifying telemetry.

2. Harness integration tests are now silently skipped in CI

harness-e2e-helper.ts was correctly updated to drop the BUILD_PREVIEW gate, but the integration tests were missed:

  • integ-tests/add-remove-harness.test.ts:9,15,90,170,197 — every describe in the file is describe.skipIf(!isPreviewBuild)
  • integ-tests/create-edge-cases.test.ts:200,202describe.skipIf(!isPreviewBuild || ...) for the harness create flow
  • integ-tests/dev-server.test.ts:178,180describe.skipIf(!isPreviewBuild || ...) for harness dev mode

build-and-test.yml runs npx vitest run --project integ without setting BUILD_PREVIEW, so all of these suites silently skip on every PR. After this PR the harness path is the default agentcore create flow, so it really needs integration coverage. Please drop the isPreviewBuild gate from these three files (the prereqs.npm/prereqs.git/hasUv checks should stay).

3. (Minor) Leftover BUILD_PREVIEW / __PREVIEW__ references

These are no longer consumed by anything, but they're confusing to a future reader. Worth either cleaning up here or in an immediate follow-up:

  • vitest.config.ts:30 still defines __PREVIEW__ as a global — feature-flags.ts no longer declares it, so this define is dead.
  • .github/workflows/e2e-tests-full.yml:33,66,115 still runs a cli-build: [preview, ga] matrix and sets BUILD_PREVIEW on the build/test steps. Since esbuild.config.mjs no longer reads it and build:preview was removed from package.json, this matrix produces two identical builds and roughly doubles full-suite runtime for no signal.
  • .github/workflows/release-main-and-preview.yml:381 sets BUILD_PREVIEW: '1' on the preview-publish build, which is now a no-op.

Happy to treat #3 as a follow-up if you'd rather keep this PR scoped, but #1 and #2 should land before merge.

Comment thread src/cli/commands/invoke/command.tsx Outdated
'invoke',
computeInvokeAttrs({
preview: isPreviewEnabled(),
preview: true,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now that preview is always on, this hardcoded preview: true plus the preview: boolean parameter on computeInvokeAttrs (in src/cli/commands/invoke/utils.ts) is dead code that still gates harness detection:

// utils.ts
const isHarness = options.preview && isHarnessInvoke(options);

Please drop the preview parameter from computeInvokeAttrs, simplify to const isHarness = isHarnessInvoke(options);, remove preview: true from this call site and from useInvokeFlow.ts, and update __tests__/utils.test.ts (the preview: false test cases are no longer meaningful).

'invoke',
computeInvokeAttrs({
preview: isPreviewEnabled(),
preview: true,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as the CLI path — this hardcoded preview: true indicates that computeInvokeAttrs's preview parameter is now dead. Drop the param from the function signature and remove it here.

// Harness features are only available in preview builds (BUILD_PREVIEW=1).
const isPreviewBuild = process.env.BUILD_PREVIEW === '1';
const baseCanRun = prereqs.npm && prereqs.git && hasAws && isPreviewBuild;
const baseCanRun = prereqs.npm && prereqs.git && hasAws;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good — but the same BUILD_PREVIEW gate is still in place in the integration tests and they're silently skipped on every CI run:

  • integ-tests/add-remove-harness.test.ts:9,15,90,170,197
  • integ-tests/create-edge-cases.test.ts:200,202 (harness create suite)
  • integ-tests/dev-server.test.ts:178,180 (harness dev suite)

build-and-test.yml runs vitest run --project integ without setting BUILD_PREVIEW, so after this PR the harness path is the default agentcore create flow but has no integ coverage in CI. Please drop the isPreviewBuild constant + skipIf from these three files (keep the prereqs.npm / prereqs.git / hasUv checks).

E2E_FILESYSTEM_SUBNET_ID: ${{ env.E2E_FILESYSTEM_SUBNET_ID }}
E2E_FILESYSTEM_SECURITY_GROUP_ID: ${{ env.E2E_FILESYSTEM_SECURITY_GROUP_ID }}
BUILD_PREVIEW: '1'
run: npx vitest run --project e2e e2e-tests/harness-bedrock.test.ts ${{ steps.changed.outputs.harness_extra }}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Minor / follow-up) BUILD_PREVIEW is no longer consumed by the build (esbuild.config.mjs and package.json's build:preview were both removed in this PR), but e2e-tests-full.yml still runs a cli-build: [preview, ga] matrix and sets BUILD_PREVIEW on the build/test steps. That now produces two identical builds and roughly doubles full-suite runtime for no signal. release-main-and-preview.yml:381 also still sets BUILD_PREVIEW: '1' for the preview publish build (also a no-op now). And vitest.config.ts:30 still defines __PREVIEW__ as a global even though feature-flags.ts no longer declares it. Worth cleaning these up here or in an immediate follow-up.

@github-actions github-actions Bot removed the agentcore-harness-reviewing AgentCore Harness review in progress label Jun 19, 2026
@github-actions github-actions Bot added size/xl PR size: XL and removed size/xl PR size: XL labels Jun 19, 2026
@agentcore-devx-automation agentcore-devx-automation Bot added the claude-security-reviewing Claude Code /security-review in progress label Jun 19, 2026
@agentcore-devx-automation

Copy link
Copy Markdown
Contributor

Claude Security Review: no high-confidence findings. (run)

@agentcore-devx-automation agentcore-devx-automation Bot removed the claude-security-reviewing Claude Code /security-review in progress label Jun 19, 2026
@github-actions github-actions Bot added size/xl PR size: XL and removed size/xl PR size: XL labels Jun 19, 2026
@agentcore-devx-automation agentcore-devx-automation Bot added the claude-security-reviewing Claude Code /security-review in progress label Jun 19, 2026
@agentcore-devx-automation

Copy link
Copy Markdown
Contributor

Claude Security Review: no high-confidence findings. (run)

@agentcore-devx-automation agentcore-devx-automation Bot removed the claude-security-reviewing Claude Code /security-review in progress label Jun 19, 2026
@github-actions github-actions Bot added size/xl PR size: XL and removed size/xl PR size: XL labels Jun 19, 2026
@agentcore-devx-automation agentcore-devx-automation Bot added the claude-security-reviewing Claude Code /security-review in progress label Jun 19, 2026
@github-actions github-actions Bot added the agentcore-harness-reviewing AgentCore Harness review in progress label Jun 19, 2026
@agentcore-devx-automation

Copy link
Copy Markdown
Contributor

Claude Security Review: no high-confidence findings. (run)

@agentcore-devx-automation agentcore-devx-automation Bot removed the claude-security-reviewing Claude Code /security-review in progress label Jun 19, 2026

@agentcore-cli-automation agentcore-cli-automation left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fixes from the previous review look good — the preview parameter is gone from computeInvokeAttrs, the isPreviewBuild gates are removed from the integ tests, and the leftover __PREVIEW__ / BUILD_PREVIEW references have been cleaned up.

One new issue with the new harness/agent path routing in create that I think should be addressed before merge — see the inline comment on HARNESS_ONLY_FLAGS.

@@ -102,17 +107,17 @@ const HARNESS_ONLY_FLAGS = [
'truncationStrategy',

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

container is harness-only (only consumed in handleCreateHarnessCLI via harnessPrimitive.parseContainerFlag(options.container) around line 236, never read in handleCreateCLI/createProjectWithAgent), but it isn't listed here. As a result, the isAgentPath(opts) && hasHarnessOnlyFlags(opts) conflict check at line ~601 doesn't fire for it, so:

agentcore create --name foo --framework Strands --container ./Dockerfile

routes to the agent path and silently drops --container instead of erroring out. Same shape as the kind of bug HARNESS_ONLY_FLAGS exists to prevent.

Fix: add 'container' (and most likely 'harnessMemory' for symmetry with --no-harness-memory) to this list.

While you're here: the hasAnyFlag chain at line ~557 is also missing container, withConfigBundle, efsAccessPointArn, efsMountPath, s3AccessPointArn, s3MountPath, and sessionStorageMountPath. In practice --name is required so users are unlikely to hit this, but agentcore create --container ./Dockerfile (no other flags) currently falls through to the TUI instead of erroring with a missing---name message. Worth syncing that list too.

@github-actions github-actions Bot removed the agentcore-harness-reviewing AgentCore Harness review in progress label Jun 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/xl PR size: XL

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Coming soon: Add Harness functionality to latest version of CLI

2 participants