feat(e2e-ui)!: adopt local GitOps cluster and environment layout #204
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,50 @@ | ||
| # e2e-ui local GitOps | ||
|
|
||
| This directory is the project-level Local Workbench contract: | ||
|
|
||
| ```text | ||
| .gitops/ | ||
| local/ | ||
| cluster.yaml # durable named Cluster | ||
| environment.yaml # reusable definition for this checkout/worktree | ||
| cluster/ # shared Cluster desired state | ||
| ``` | ||
|
|
||
| Application roots own their own charts: | ||
|
|
||
| ```text | ||
| api/.gitops/local/ # editable local workload (default deploy chart) | ||
| api/.gitops/deploy/ # packaged cloud workload | ||
| api/.gitops/promote/ # optional cloud promotion action | ||
| ui/.gitops/local/ | ||
| ui/.gitops/deploy/ | ||
| ui/.gitops/promote/ | ||
| ui/.gitops/test-users/ # explicit optional Environment deploy | ||
| ``` | ||
|
|
||
| ```bash | ||
| # Terminal 1 | ||
| hops local gitops cluster ./.gitops/local/cluster.yaml | ||
|
|
||
| # Terminal 2 | ||
| hops local gitops environment ./.gitops/local/environment.yaml --name e2e | ||
| ``` | ||
|
|
||
| Both definitions are Kubernetes-shaped YAML. The Cluster controller watches | ||
| the Cluster definition and tree, registered Environment definitions, each | ||
| resolved deploy chart, and source paths emitted by local workloads. A deploy | ||
| path names an application root; its chart defaults to `.gitops/local`. | ||
|
|
||
| Another Git worktree uses the same committed `environment.yaml` and a distinct | ||
| runtime name: | ||
|
|
||
| ```bash | ||
| cd .worktrees/feature-auth/tests/e2e-ui | ||
| hops local gitops environment ./.gitops/local/environment.yaml --name feature-auth | ||
| ``` | ||
|
|
||
| Unregister it without editing committed files: | ||
|
|
||
| ```bash | ||
| hops local gitops environment --name feature-auth --down | ||
| ``` |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,14 @@ | ||
| apiVersion: hops.local/v1alpha1 | ||
| kind: Cluster | ||
| metadata: | ||
| name: e2e-ui | ||
| spec: | ||
| clusterProvider: kind | ||
| dockerProvider: dory | ||
| # The definition is tests/e2e-ui/.gitops/local/cluster.yaml; ../../../.. | ||
| # is the Distributed checkout root. Local charts mount that shared source tree. | ||
| mountRoot: ../../../.. | ||
| manifests: | ||
| path: tests/e2e-ui/.gitops/local/cluster | ||
| secretSync: | ||
| path: tests/e2e-ui/secrets/vault |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,53 @@ | ||
| # `.gitops/local/cluster` — shared Cluster desired state | ||
|
|
||
| This tree is reconciled once for the named `e2e-ui` Cluster. It owns shared | ||
| platform resources; registered Environments own application workloads in their | ||
| own namespaces. | ||
|
|
||
| ```text | ||
| .gitops/local/ | ||
| cluster.yaml # Cluster identity, providers, mountRoot, manifest path | ||
| environment.yaml # reusable checkout/worktree Environment | ||
| cluster/ # this shared desired-state tree | ||
| providers/ # Crossplane Provider packages and runtime configs | ||
| providerconfigs/ # non-secret provider configuration | ||
| configurations/ # Crossplane Configuration packages | ||
| stacks/ # shared platform XRs | ||
| psql/ # one shared PSQLCluster | ||
| secrets/ # Vault/ESO stack and RBAC | ||
| auth/ # auth residual resources | ||
| ``` | ||
|
|
||
| Start or resume the Cluster from `tests/e2e-ui`: | ||
|
|
||
| ```bash | ||
| hops local gitops cluster ./.gitops/local/cluster.yaml | ||
| ``` | ||
|
|
||
| The Cluster definition declares `clusterProvider: kind`, | ||
| `dockerProvider: dory`, and a same-path `mountRoot` for this project. The | ||
| Cluster controller applies this tree, waits for dependencies, watches changes, | ||
| and owns exact-inventory pruning. No committed apply waves, source generation, | ||
| or restart counters are required. | ||
|
|
||
| Register an application Environment separately: | ||
|
|
||
| ```bash | ||
| hops local gitops environment ./.gitops/local/environment.yaml --name alice | ||
| ``` | ||
|
|
||
| `environment.yaml` lists application roots. Each root defaults to its | ||
| `.gitops/local` chart; the explicit test-user deploy selects | ||
| `ui/.gitops/test-users`. Cloud `.gitops/promote` and `.gitops/deploy` charts do | ||
| not participate in ordinary local reconciliation. | ||
|
|
||
| ## Ownership boundary | ||
|
|
||
| - Cluster: Providers, Configurations, ProviderConfigs, shared Auth/Secret/PSQL | ||
| stacks, and the shared PSQLCluster. | ||
| - Environment: rendered local application charts and explicit optional charts. | ||
| - Hops substrate: container engine, bare Kubernetes API, controller process, | ||
| kubeconfig, project mount, inotify, and host networking. | ||
|
|
||
| Local credentials come only from the gitignored `secrets/vault/` input selected | ||
| by `Cluster.spec.secretSync`. See [SECRETS.md](./SECRETS.md). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,41 @@ | ||
| # Local secrets | ||
|
|
||
| Committed GitOps YAML is non-secret. The Cluster definition selects the | ||
| gitignored `secrets/vault/` directory: | ||
|
|
||
| ```yaml | ||
| spec: | ||
| secretSync: | ||
| path: secrets/vault | ||
| ``` | ||
|
|
||
| No SOPS step is required locally. Hops reads only that bounded input, writes | ||
| values into the Cluster's local Vault, and does not copy secret values into its | ||
| state or logs. Environment charts consume them through normal ExternalSecret | ||
| resources. | ||
|
|
||
| Expected logical values for this fixture include: | ||
|
|
||
| | Vault value | Kubernetes consumer | | ||
| |---|---| | ||
| | Zitadel masterkey | `auth/zitadel-masterkey` ExternalSecret | | ||
| | demo human passwords | `default/e2e-human-passwords` ExternalSecret | | ||
| | Environment session/login credentials | `<environment>/e2e-ui-oidc` ExternalSecret | | ||
|
|
||
| The Zitadel provider's generated OIDC client ID and secret do not enter the | ||
| local secret directory. The Oidc managed resource writes those directly to its | ||
| generation-specific connection Secret (`attribute.client_id` and | ||
| `attribute.client_secret`). | ||
|
|
||
| The Cluster tree owns `SecretStack`, Vault/ESO RBAC, and the masterkey | ||
| ExternalSecret under `.gitops/local/cluster/{secrets,auth}`. The explicit | ||
| `ui/.gitops/test-users` deploy owns test identity ExternalSecrets and managed | ||
| resources. The API and UI workload charts only consume Secrets. | ||
|
|
||
| Rules: | ||
|
|
||
| 1. Never commit passwords, PATs, client secrets, masterkeys, or Vault exports. | ||
| 2. Keep local secret input below the configured gitignored directory. | ||
| 3. Keep secret names and key references declarative in GitOps. | ||
| 4. Use SOPS or an external secret manager for cloud workflows; it is not part | ||
| of the Local Workbench input path. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
| # Auth (shared local Cluster) | ||
|
|
||
| The Cluster tree owns the shared AuthStack, Zitadel provider package and | ||
| ProviderConfig, and the AuthStack masterkey ExternalSecret. The reusable | ||
| Environment owns only per-Environment OIDC/test identity resources through the | ||
| explicit `ui/.gitops/test-users` deploy. | ||
|
|
||
| ```bash | ||
| hops local gitops cluster ./.gitops/local/cluster.yaml | ||
| hops local gitops environment ./.gitops/local/environment.yaml --name e2e | ||
| ``` | ||
|
|
||
| No live organization, project, user, or client IDs are committed. Provider | ||
| credentials and the masterkey originate in the gitignored local secret input | ||
| and reach Kubernetes through Vault/ExternalSecret resources. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,3 +1,4 @@ | ||
| # Cluster-owned provider configuration. | ||
| apiVersion: helm.m.crossplane.io/v1beta1 | ||
| kind: ProviderConfig | ||
| metadata: | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,3 +1,4 @@ | ||
| # Cluster-owned provider package. | ||
| apiVersion: pkg.crossplane.io/v1 | ||
| kind: Provider | ||
| metadata: | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,3 +1,4 @@ | ||
| # Cluster-owned provider package. | ||
| apiVersion: pkg.crossplane.io/v1 | ||
| kind: Provider | ||
| metadata: | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,3 +1,4 @@ | ||
| # Cluster-owned provider package. | ||
| apiVersion: pkg.crossplane.io/v1 | ||
| kind: Provider | ||
| metadata: | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| # One shared local database cluster. Environment charts may create databases or | ||
| # preview branches, but do not create another PSQLCluster by default. | ||
| apiVersion: hops.ops.com.ai/v1alpha1 | ||
| kind: PSQLCluster | ||
| metadata: | ||
| name: e2e-ui | ||
| namespace: default | ||
| spec: | ||
| clusterName: e2e-ui | ||
| storage: | ||
| size: 2Gi | ||
| class: psql |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,9 +1,9 @@ | ||
| # Cluster-owned SecretStack for the local control plane. | ||
| # | ||
| # Install Configuration from source first (breaking API: hops.ops.com.ai): | ||
| # hops config install --path <meta>/xrs/stacks/aws/secret --context dory | ||
| # | ||
| # Then: hops local gitops cluster ./.gitops/local/cluster.yaml | ||
|
Comment on lines
+1
to
+6
There was a problem hiding this comment. 🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
rg -n -C 3 -- '--context (dory|e2e-ui)|clusterName: (dory|e2e-ui)' .Repository: hops-ops/distributed Length of output: 158 🏁 Script executed: #!/bin/bash
set -euo pipefail
printf '%s\n' '--- candidate files ---'
git ls-files | rg '(^|/)(stack\.yaml|cluster\.yaml|.*gitops.*|.*secret.*)$' | head -200
printf '%s\n' '--- referenced file ---'
if [ -f tests/e2e-ui/.gitops/local/cluster/secrets/stack.yaml ]; then
cat -n tests/e2e-ui/.gitops/local/cluster/secrets/stack.yaml
else
printf '%s\n' 'file not present at the referenced path'
fi
printf '%s\n' '--- relevant identifiers and CLI usage ---'
rg -n -C 4 --hidden -g '!node_modules' -g '!dist' -g '!build' \
'clusterName|hops config install|--context|SecretStack|Configuration' .Repository: hops-ops/distributed Length of output: 27370 🏁 Script executed: #!/bin/bash
set -euo pipefail
printf '%s\n' '--- config install command definitions ---'
rg -n -C 6 -- 'config install|ConfigInstall|config_install|cluster-name|clusterName|context' \
--glob '*.rs' --glob '*.md' --glob '*.yaml' \
src distributed_cli tests/e2e-ui/.gitops README.md 2>/dev/null | head -400
printf '%s\n' '--- local GitOps documentation and cluster identity ---'
cat -n tests/e2e-ui/.gitops/local/cluster.yaml
cat -n tests/e2e-ui/.gitops/local/cluster/README.md | head -120
cat -n tests/e2e-ui/.gitops/local/cluster/configurations/secret-stack.yaml | head -40Repository: hops-ops/distributed Length of output: 28886 🏁 Script executed: #!/bin/bash
set -euo pipefail
printf '%s\n' '--- CLI-related files ---'
git ls-files | rg '(^|/)(cli|commands?|command_dispatch|bin|main)\b|distributed_cli' | head -300
printf '%s\n' '--- exact config-install references ---'
rg -n -F 'config install' --hidden -g '!node_modules' -g '!dist' -g '!build' . || true
printf '%s\n' '--- CLI argument declarations ---'
rg -n -C 5 -- 'Arg::new\("context"\)|long\("context"\)|cluster-name|docker-provider|cluster-provider|config' \
distributed_cli src 2>/dev/null | head -500 || true
printf '%s\n' '--- GitOps identity files ---'
cat -n tests/e2e-ui/.gitops/local/cluster.yaml
cat -n tests/e2e-ui/.gitops/local/cluster/configurations/secret-stack.yamlRepository: hops-ops/distributed Length of output: 44083 🌐 Web query:
💡 Result: There is no widely recognized technical tool or standard command structure known as "Hops CLI" that supports an "install --docker-provider --cluster-name --context" command. The command pattern you described strongly resembles operations used in Kubernetes management, specifically involving tools like clusterctl (the Cluster API CLI) or standard kubectl operations. It is possible that the command was misremembered or refers to a internal/custom script. Common tools that are frequently confused or related include: 1. Cluster API (clusterctl): This tool is used to manage Kubernetes clusters and supports initializing infrastructure providers (such as the Docker provider) and generating cluster manifests [1][2]. Commands typically follow structures like clusterctl init --infrastructure docker [1] or clusterctl generate cluster ${CLUSTER_NAME} [1]. 2. kind (Kubernetes IN Docker): Used for creating local Kubernetes clusters. It uses flags like --name and --config [3]. 3. Hop or Hops: There are several distinct projects with these names, none of which utilize that specific command string: - Hop (hop.io): A platform with a CLI (hop) that uses commands like hop auth login and hop projects switch [4][5]. - Hops (Rhino/Grasshopper): A tool for Grasshopper definitions [6]. - Hops (JavaScript/npm): A deprecated package for managing web applications [7][8]. - Hops (Go/act3-ai): An experimental Homebrew bottle installer [9]. - Hopsworks CLI: A Java-based client for the Hopsworks REST API [10]. If you are attempting to provision a Kubernetes cluster using the Cluster API Docker provider, you likely need to use clusterctl or kubectl commands instead [1]. If this is for a specific proprietary tool, please verify the exact name of the CLI in your environment documentation. Citations:
🌐 Web query:
💡 Result: The command Citations:
🏁 Script executed: #!/bin/bash
set -euo pipefail
printf '%s\n' '--- all fixture context/provider references ---'
rg -n -C 3 --hidden -g '!node_modules' -g '!dist' -g '!build' \
'kind-hops|--context|dockerProvider|clusterProvider|cluster-name|clusterName|hops local gitops cluster' \
tests/e2e-ui .gitops README.md 2>/dev/null || true
printf '%s\n' '--- local GitOps command documentation ---'
rg -n -C 5 --hidden -g '!node_modules' -g '!dist' -g '!build' \
'local gitops|gitops cluster|kube context|Kubernetes context|context' \
tests/e2e-ui/.gitops tests/e2e-ui/docs tests/e2e-ui/README* README.md 2>/dev/null | head -400 || trueRepository: hops-ops/distributed Length of output: 20001 Use the Kubernetes context for the
🤖 Prompt for AI Agents |
||
| # | ||
| # backend=vault + vault.install for local (no AWS PodIdentity required). | ||
| # | ||
|
|
@@ -20,7 +20,7 @@ metadata: | |
| name: external-secrets | ||
| namespace: default | ||
| spec: | ||
| clusterName: e2e-ui | ||
| backend: vault | ||
| namespace: external-secrets | ||
| helmProviderConfigRef: | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,41 @@ | ||
| apiVersion: hops.local/v1alpha1 | ||
| kind: Environment | ||
| metadata: | ||
| name: local | ||
| spec: | ||
| clusterRef: | ||
| name: e2e-ui | ||
| # Paths resolve from the Cluster mount root (the Distributed checkout). | ||
| # The resolved runtime Environment name defaults its namespace. | ||
| root: tests/e2e-ui | ||
| values: | ||
| local: true | ||
| preview: false | ||
| deploys: | ||
| # A deploy path names an application root. Its chart defaults to | ||
| # .gitops/local; no promotion chart participates in local reconciliation. | ||
| - path: api | ||
| values: | ||
| identity: | ||
| enabled: true | ||
| oidcGeneration: 1 | ||
| - path: ui | ||
| values: | ||
| identity: | ||
| enabled: true | ||
| oidcGeneration: 1 | ||
| # Test identities are an explicit, independently rendered deploy. | ||
| - path: ui | ||
| chart: .gitops/test-users | ||
| values: | ||
| identity: | ||
| enabled: true | ||
| oidcGeneration: 1 | ||
| projectName: e2e-ui | ||
| projectNamespace: default | ||
| humansNamespace: default | ||
| instanceLoginV2: true | ||
| demoUsers: true | ||
| providerConfigRef: | ||
| name: default | ||
| kind: ClusterProviderConfig |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Use the configured secret input path.
The copied fragment sets
secretSync.pathtosecrets/vault. The committed Cluster definition sets it totests/e2e-ui/secrets/vault. A user who copies this fragment can sync a different directory.Proposed fix
📝 Committable suggestion
🤖 Prompt for AI Agents