-
Notifications
You must be signed in to change notification settings - Fork 1
feat(cli): native blockstor CLI speaking Kubernetes directly #181
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
Andrei Kvapil (kvaps)
wants to merge
22
commits into
main
Choose a base branch
from
feat/blockstor-cli
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
22 commits
Select commit
Hold shift + click to select a range
6f71d95
feat(cli): printer columns on every CRD + colour classification
kvaps dd5dc84
feat(cli): command registry with upstream-compatible aliases
kvaps ca6574a
feat(cli): metav1.Table renderer with layout-safe colouring
kvaps ab96499
feat(cli): resource list view with the derived state column
kvaps 0e2a4af
feat(cli): blockstor binary — dispatch, exit codes, machine output
kvaps f6a42a5
feat(cli): the remaining list views
kvaps 687d3ab
feat(cli): property verbs, definition create/delete, controller version
kvaps 62616db
feat(cli): node, volume-definition, resource, snapshot and group writes
kvaps f2cf7e8
feat(cli): property verbs on every noun that carries a property bag
kvaps 126b2ad
feat(cli): storage-pool and volume-group lifecycle
kvaps cbfda52
feat(cli): per-replica verbs — toggle-disk, activate, deactivate
kvaps 329a372
feat(cli): node lifecycle — evacuate, evict, restore, lost, info
kvaps 293d56e
feat(cli): drbd-options on controller, definitions and volumes
kvaps 73a0a0e
feat(cli): autoplace, spawn-resources and adjust
kvaps 47a48e8
feat(cli): encryption passphrase, and the limit of a CRD-only client
kvaps f89d88f
feat(cli): clone, modify, restore, create-multiple and size queries
kvaps 3f50385
feat(cli): help output, and document what Kubernetes does not hold
kvaps 6b6a401
fix(cli): shortfall contract per verb, and resize guards
kvaps de41705
refactor(cli): drop error-reports from the command surface
kvaps e20b56f
feat(cli): enter-passphrase verifies against the Secret and succeeds
kvaps 7ec29b1
fix(cli): size bounds, faulty filtering, flag handling from review
kvaps ed15a89
fix(cli): review round two — write ordering, output gaps, dead paths
kvaps File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,103 @@ | ||
| // SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| /* | ||
| Copyright 2026 Cozystack contributors. | ||
|
|
||
| Licensed under the Apache License, Version 2.0 (the "License"); | ||
| you may not use this file except in compliance with the License. | ||
| You may obtain a copy of the License at | ||
|
|
||
| http://www.apache.org/licenses/LICENSE-2.0 | ||
|
|
||
| Unless required by applicable law or agreed to in writing, software | ||
| distributed under the License is distributed on an "AS IS" BASIS, | ||
| WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| See the License for the specific language governing permissions and | ||
| limitations under the License. | ||
| */ | ||
|
|
||
| package v1alpha1_test | ||
|
|
||
| import ( | ||
| "os" | ||
| "path/filepath" | ||
| "testing" | ||
|
|
||
| "sigs.k8s.io/yaml" | ||
|
|
||
| apiextv1 "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1" | ||
| ) | ||
|
|
||
| // Every CRD carries printer columns so `kubectl get` is useful on its | ||
| // own and the CLI can lean on server-side table printing. Without them | ||
| // `kubectl get resources` shows NAME/AGE and nothing else — an operator | ||
| // debugging a stuck replica learns nothing from it. | ||
| // | ||
| // The columns are also a compatibility surface: the CLI's tables and | ||
| // kubectl's tables should not disagree about what a resource's state | ||
| // is, so the set is pinned here rather than left to drift. | ||
| func TestCRDsCarryPrinterColumns(t *testing.T) { | ||
| t.Parallel() | ||
|
|
||
| want := map[string][]string{ | ||
| "blockstor.cozystack.io_nodes.yaml": {"Type", "Address", "Status", "Age"}, | ||
| "blockstor.cozystack.io_storagepools.yaml": {"Node", "Pool", "Provider", "Free-KiB", "Total-KiB", "Age"}, | ||
| "blockstor.cozystack.io_resourcedefinitions.yaml": {"Group", "Port", "Layers", "Age"}, | ||
| "blockstor.cozystack.io_resources.yaml": {"Definition", "Node", "Pool", "Node-ID", "Port", "State", "In-Use", "Age"}, | ||
| "blockstor.cozystack.io_snapshots.yaml": {"Definition", "Snapshot", "Nodes", "Age"}, | ||
| "blockstor.cozystack.io_resourcegroups.yaml": {"Place-Count", "Storage-Pool", "Layers", "Age"}, | ||
| } | ||
|
|
||
| for file, wantCols := range want { | ||
| t.Run(file, func(t *testing.T) { | ||
| t.Parallel() | ||
|
|
||
| got := printerColumnNames(t, file) | ||
| if len(got) != len(wantCols) { | ||
| t.Fatalf("printer columns = %v, want %v", got, wantCols) | ||
| } | ||
|
|
||
| for i := range wantCols { | ||
| if got[i] != wantCols[i] { | ||
| t.Errorf("column[%d] = %q, want %q (full set %v)", i, got[i], wantCols[i], got) | ||
| } | ||
| } | ||
| }) | ||
| } | ||
| } | ||
|
|
||
| // printerColumnNames reads the generated CRD and returns the served | ||
| // version's printer-column names in declaration order. | ||
| func printerColumnNames(t *testing.T, file string) []string { | ||
| t.Helper() | ||
|
|
||
| data, err := os.ReadFile(filepath.Join("..", "..", "config", "crd", "bases", file)) | ||
| if err != nil { | ||
| t.Fatalf("read CRD: %v (run `make manifests`)", err) | ||
| } | ||
|
|
||
| var crd apiextv1.CustomResourceDefinition | ||
|
|
||
| err = yaml.Unmarshal(data, &crd) | ||
| if err != nil { | ||
| t.Fatalf("parse CRD %s: %v", file, err) | ||
| } | ||
|
|
||
| for i := range crd.Spec.Versions { | ||
| version := &crd.Spec.Versions[i] | ||
| if !version.Served { | ||
| continue | ||
| } | ||
|
|
||
| names := make([]string, 0, len(version.AdditionalPrinterColumns)) | ||
| for j := range version.AdditionalPrinterColumns { | ||
| names = append(names, version.AdditionalPrinterColumns[j].Name) | ||
| } | ||
|
|
||
| return names | ||
| } | ||
|
|
||
| t.Fatalf("CRD %s has no served version", file) | ||
|
|
||
| return nil | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,134 @@ | ||
| // SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| /* | ||
| Copyright 2026 Cozystack contributors. | ||
|
|
||
| Licensed under the Apache License, Version 2.0 (the "License"); | ||
| you may not use this file except in compliance with the License. | ||
| You may obtain a copy of the License at | ||
|
|
||
| http://www.apache.org/licenses/LICENSE-2.0 | ||
|
|
||
| Unless required by applicable law or agreed to in writing, software | ||
| distributed under the License is distributed on an "AS IS" BASIS, | ||
| WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| See the License for the specific language governing permissions and | ||
| limitations under the License. | ||
| */ | ||
|
|
||
| // Command blockstor is the storage CLI. | ||
| // | ||
| // It speaks the Kubernetes API directly — the CRDs are the source of | ||
| // truth, so there is no control-plane hop between the operator and the | ||
| // data. Reading straight from the API also sidesteps the informer-cache | ||
| // lag the multi-replica REST apiserver has to retry around: this client | ||
| // sees its own writes. | ||
| // | ||
| // The grammar mirrors the client operators already use, long and short: | ||
| // | ||
| // blockstor resource list blockstor r l | ||
| // blockstor storage-pool list blockstor sp l | ||
| // blockstor node list --nodes n1 blockstor n l -n n1 | ||
| // | ||
| // Exit codes follow the same convention as the client it replaces: 0 | ||
| // success, 2 a client-side rejection (unknown command or flag), 10 an | ||
| // API-level failure. | ||
| package main | ||
|
|
||
| import ( | ||
| "context" | ||
| "fmt" | ||
| "os" | ||
| "strings" | ||
|
|
||
| ctrlclient "sigs.k8s.io/controller-runtime/pkg/client" | ||
| "sigs.k8s.io/controller-runtime/pkg/client/config" | ||
|
|
||
| corev1 "k8s.io/api/core/v1" | ||
| "k8s.io/apimachinery/pkg/runtime" | ||
|
|
||
| crdv1alpha1 "github.com/cozystack/blockstor/api/v1alpha1" | ||
| "github.com/cozystack/blockstor/pkg/store" | ||
| storek8s "github.com/cozystack/blockstor/pkg/store/k8s" | ||
|
|
||
| "github.com/cozystack/blockstor/internal/cli" | ||
| ) | ||
|
|
||
| func main() { | ||
| app := &cli.App{ | ||
| Out: os.Stdout, | ||
| Err: os.Stderr, | ||
| StoreFor: openStore, | ||
| KubeFor: openKube, | ||
| } | ||
|
|
||
| os.Exit(app.Run(context.Background(), os.Args[1:])) | ||
| } | ||
|
|
||
| // openStore builds the CRD-backed store from the ambient kubeconfig | ||
| // (KUBECONFIG, ~/.kube/config, or the in-cluster service account). | ||
| // | ||
| // The client is deliberately NOT cached: a cache would reintroduce the | ||
| // read-your-writes lag the apiserver has to defend against, and a CLI | ||
| // process that lists once has nothing to gain from an informer. | ||
| func openStore(ctx context.Context) (store.Store, error) { | ||
| c, _, err := openKube(ctx) | ||
| if err != nil { | ||
| return nil, err | ||
| } | ||
|
|
||
| return storek8s.New(c), nil | ||
| } | ||
|
|
||
| // openKube builds the raw client plus the namespace the controller | ||
| // runs in, for the few commands that reach objects outside the CRD | ||
| // surface (the cluster passphrase lives in a Secret). | ||
| func openKube(context.Context) (ctrlclient.Client, string, error) { | ||
| cfg, err := config.GetConfig() | ||
| if err != nil { | ||
| return nil, "", fmt.Errorf("load kubeconfig: %w", err) | ||
| } | ||
|
|
||
| scheme := runtime.NewScheme() | ||
|
|
||
| err = crdv1alpha1.AddToScheme(scheme) | ||
| if err != nil { | ||
| return nil, "", fmt.Errorf("register scheme: %w", err) | ||
| } | ||
|
|
||
| err = corev1.AddToScheme(scheme) | ||
| if err != nil { | ||
| return nil, "", fmt.Errorf("register core scheme: %w", err) | ||
| } | ||
|
|
||
| c, err := ctrlclient.New(cfg, ctrlclient.Options{Scheme: scheme}) | ||
| if err != nil { | ||
| return nil, "", fmt.Errorf("connect to the Kubernetes API: %w", err) | ||
| } | ||
|
|
||
| return c, namespace(), nil | ||
| } | ||
|
|
||
| // namespace resolves where the controller's own objects live: the | ||
| // in-cluster service-account namespace when running as a pod, the | ||
| // BLOCKSTOR_NAMESPACE override otherwise, and the deployment default | ||
| // last. | ||
| func namespace() string { | ||
| if ns := os.Getenv("BLOCKSTOR_NAMESPACE"); ns != "" { | ||
| return ns | ||
| } | ||
|
|
||
| data, err := os.ReadFile(serviceAccountNamespaceFile) | ||
| if err == nil && len(data) > 0 { | ||
| return strings.TrimSpace(string(data)) | ||
| } | ||
|
|
||
| return defaultNamespace | ||
| } | ||
|
|
||
| // serviceAccountNamespaceFile is where kubelet projects a pod's own | ||
| // namespace. | ||
| const serviceAccountNamespaceFile = "/var/run/secrets/kubernetes.io/serviceaccount/namespace" | ||
|
|
||
| // defaultNamespace matches the deployment manifests. | ||
| const defaultNamespace = "blockstor-system" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
Repository: cozystack/blockstor
Length of output: 275
🏁 Script executed:
Repository: cozystack/blockstor
Length of output: 35857
🏁 Script executed:
Repository: cozystack/blockstor
Length of output: 274
🏁 Script executed:
Repository: cozystack/blockstor
Length of output: 37084
🏁 Script executed:
Repository: cozystack/blockstor
Length of output: 18508
🌐 Web query:
Kubernetes CustomResourceDefinition additionalPrinterColumns JSONPath array result type mismatch printer-column omitted💡 Result:
In Kubernetes CustomResourceDefinitions (CRDs), the
additionalPrinterColumnsconfiguration is designed to extract values from a custom resource to display inkubectltable output [1][2]. If a JSONPath expression targets an array, the observed behavior—where the column may appear empty, behave inconsistently, or fail to display multiple elements—is a known area of historical complexity in Kubernetes [3][4][5]. Key Technical Context: 1. Historical Limitation: Early versions of Kubernetes often restricted or inconsistently handled JSONPaths that returned array types inadditionalPrinterColumns[3][5]. Historically, the system might have only evaluated and printed the first matching result, or failed to handle the array-to-string conversion expected for a flat table column [3][5]. 2. Improved Support: Subsequent updates, such as the changes introduced to improve support for more complex JSONPaths, have allowed for better handling of paths that return multiple values [4]. However, these are still subject to the requirement that the resulting data must be representable as a string within the table cell [1][6]. 3. Requirement for Simple Paths: While documentation often refers to "simple JSON paths" [1][6], "simple" in this context usually refers to the structure of the path itself rather than the complexity of the data it points to. If the JSONPath evaluates to an object or an array,kubectlmust effectively serialize that data into a string format to display it in a table column [7]. Troubleshooting Steps: If you are experiencing a type mismatch or missing data: - Ensure thetypedefined in yourCustomResourceColumnDefinition(e.g.,string,integer) matches the expected output of your JSONPath [1][6]. If the path returns an array, defining the column type asstringis generally required, andkubectlwill typically join the array elements into a comma-separated string [7]. - Verify the JSONPath expression is valid. For example, using[*]can help ensure you are selecting all elements, but be mindful of the performance cost of complex path evaluations on the API server [4][5]. - If the output remains empty or missing, it may indicate that the API server is unable to resolve the path or that the resulting data type is incompatible with the declared column type. As a workaround, some users find it necessary to manage this data at the application layer by creating specific, scalar fields in the CRD status (e.g., a pre-computed string field) rather than relying on dynamic JSONPath extraction for complex data structures [4][5].Citations:
🌐 Web query:
Kubernetes API machinery printer columns JSONPath array result type string omit💡 Result:
In Kubernetes Custom Resource Definition (CRD) additional printer columns, the support for arrays in JSONPath is limited and often yields unexpected results. When you use a JSONPath that targets an array (e.g.,.spec.items[*].name), the Kubernetes server-side printing logic typically returns only the first match rather than the full array [1][2]. This behavior is a known limitation that persists in many versions, where the system does not automatically join or format array elements for display in the column [1][2]. Regarding the "omit" or "empty" result type: 1. Handling Missing Data: If a field is missing, the column may appear empty [3][4]. While Kubernetes JSONPath generally allows accessing fields, you cannot explicitly configure an "omit" property in the CRD definition to hide the column if the value is missing. The visibility of a column is instead governed by its priority [5][6]. 2. Column Priority: You can use the priority field in the CustomResourceColumnDefinition [5]. Columns with a priority greater than 0 are considered "wide" columns and are only displayed when the user explicitly requests wider output (e.g., via -o wide), effectively omitting them from standard output if space is limited [5][6]. 3. Escaping: If you are encountering empty results for fields that you believe exist, ensure you are correctly escaping special characters (such as periods or slashes in labels) using backslashes (e.g.,.metadata.labels.key.name) [3][7][4]. Single quotes are sometimes required around the path for specific characters to be parsed correctly [3]. Because there is no native "omit if empty" feature for printer columns, if your goal is to handle complex array data or conditional visibility, the recommended workaround is often to define a specific status field in your API that pre-formats the data (e.g., as a single string) [2]. This allows you to point the printer column to a simple string field rather than an array, ensuring consistent output [2].
Citations:
🌐 Web query:
Kubernetes API printer columns jsonpath arrays type string join comma kubectl get -o wide💡 Result:
In Kubernetes, CustomResourceDefinition (CRD)
additionalPrinterColumnsdo not support complex JSONPath templates or native array joining [1][2][3]. ThejsonPathfield in a CRD specification is restricted to simple JSONPath expressions that must start with a dot (.) and point to a specific field [2][3]. If you use a JSONPath that returns an array (e.g.,.spec.items[*]), Kubernetes will typically display only the first element of that array rather than joining the elements with commas [1][4]. For scenarios where you need to display joined, formatted, or combined data (like X/Y or comma-separated lists) in a table format, you have the following options: 1. Usekubectl -o custom-columns: Unlike CRD printer columns, thekubectlcustom-columns output format supports more flexible JSONPath templates [5]. You can command line the desired output without modifying the CRD: kubectl get -o custom-columns=NAME:.metadata.name,MY_COL:.spec.items[*] 2. Process in the application or controller: Since CRD printer columns are limited, the community standard approach is to create a specific, pre-formatted string field in your CRD'sstatusblock (e.g.,status.itemsSummary). Your controller or operator then computes this joined string (e.g., usingstrings.Join()) and updates the object status [1]. You then point your CRDadditionalPrinterColumnsto that pre-computed string field [1]. 3. Use alternative output formats: If you need complex manipulation or formatting of lists for human-readable output, consider usingkubectl -o jsonorkubectl -o yamlpiped into tools likejq(e.g.,kubectl get <resource> -o json | jq '.items[].spec.items | join(",")'), which natively supports advanced data transformation and string joining [6].Citations:
Use a scalar summary field for populated array printer columns.
These paths target array-valued CRD fields, so the
Layers/Nodescolumns can render inconsistent or omitted values in kubectl table output. Replace them with a controller-provided string summary/status field, or remove the columns.📍 Affects 3 files
config/crd/bases/blockstor.cozystack.io_resourcedefinitions.yaml#L24-L26(this comment)config/crd/bases/blockstor.cozystack.io_resourcegroups.yaml#L24-L26config/crd/bases/blockstor.cozystack.io_snapshots.yaml#L24-L26🤖 Prompt for AI Agents
Source: MCP tools