feat: persist index credentials in the OS keyring (sysand auth login/logout/status/whoami)

[consideRatio]

Non-AI writeup - I just wanted to see what came out of following the planned feature, and enable live testing a UX with it.

It was mostly fable 5 doing the work, fallback to opus 8 some times i figure.

---

Implements credential storage and the sysand auth command family, v1 of the design settled in #437 (the reviewed plan lives in this comment and is imported as design/credential-storage.md in this branch's first commit).

A note on how this PR came to be: it is a one-shot, agent-driven implementation of that plan. The thinking work happened in the plan itself (several adversarial review passes, documented in the issue); the code was then produced in one pass of small reviewed increments, each verified on the integrated branch. If the direction is wrong, discarding this costs tokens, not accumulated human design work, so please review the plan first and treat the code as its consequence. Draft until a maintainer wants to take it forward.

The server-side counterpart (the v1/whoami endpoint) is already deployed on sysand.com (a validating login against it probes v1/whoami).

What this delivers

Store an index credential once with sysand auth login, and sysand reuses it across runs on Windows, macOS, and Linux, with the secret held by the OS keyring, never a plaintext file. Bearer-only v1; SYSAND_CRED_* env vars keep working unchanged and take precedence (CI overrides an interactive login).

Command surface: auth login / logout / status

• sysand auth login [INDEX_URL] [--token-stdin]
  • No URL resolves the default-index chain (SYSAND_DEFAULT_INDEX, config default = true, else https://sysand.com); more than one configured default errors and asks for an explicit URL. The resolved index is always echoed before any secret entry. There is deliberately no per-subcommand --default-index flag: it would only duplicate the positional argument.
  • Secret via hidden prompt (Enter token for ...:, rpassword) or --token-stdin (trims exactly one trailing newline); never an inline argument. No TTY and no --token-stdin fails fast instead of hanging. Empty token errors.
  • Re-login over an existing entry prints a replacing notice before the write.
  • On a host with no usable keyring backend, login refuses to persist and prints exact SYSAND_CRED_* lines with the secret as a literal <token> placeholder (never echoed; asserted by test).
• sysand auth logout [INDEX_URL]: removes a stored login (same default-index resolution, echo, and clear missing-entry error).
• sysand auth status: one unified view of everything sysand will authenticate with. Stored entries (key in the exact auth logout <key> form, covered globs, subject and token prefix when a validating login ran, expires in N days / expired, SYSAND_CRED_* shadowing) and env entries, tagged by source; each stored entry shows its persisted validation claim (validated (read/api) or a warn-styled not validated); absence negatives print only when nothing is configured at all; entries covering the resolved default index carry a dim (default index) marker. Never shows secrets. Output is styled with the CLI's existing anstyle tokens and aligned to the same 12-column gutter as the rest of the CLI (plain when piped or under NO_COLOR).
• sysand auth whoami [INDEX_URL]: query-only live identity check against v1/whoami, using the same credential selection as the runtime (env over stored, and the output names which source was used). Exit 0 only on an accepted credential; rejected/unreachable/rate-limited get distinct nonzero messages. Never writes the store. Errors clearly when the index advertises no API.
• URL-template index locations (for example a GitLab repository-files-API index, https://gitlab.com/api/v4/projects/<id>/repository/files/{path}/raw?ref=index) are accepted as login/logout/whoami targets: the credential is scoped to the template's literal prefix, and template keys are normalized so login/logout/status round-trip.

Storage: single keyring blob, fail-closed, locked

• All persisted credentials live in one OS-keyring entry (service sysand, account credentials) holding a versioned JSON blob of records {key, globs, scheme, secret, expires_at?, subject?, token_name?, token_prefix?}.
• Fail-closed codec: unknown fields round-trip (serde(flatten)), a parse failure surfaces "credential store unreadable" instead of silently clobbering stored credentials. A pre-existing older blob is proven to still parse.
• Read-modify-write is guarded by a cross-process advisory file lock (fd-lock) at a per-user path, bounded wait; never an existence-based lock file.
• Windows blob cap (~2.5 KB, measured in UTF-16 bytes) yields a clear "store full / token too large" error instead of truncation.
• Error taxonomy: backend absent (env fallback allowed) vs locked/denied (surfaced with an unlock suggestion and the SYSAND_CRED_* fallback named).
• Cargo feature layout: record types, codec, and the CredentialStore trait are unconditional core; the OS-keyring backend sits behind a new non-default keyring feature (enabled by the CLI; the js/wasm binding stays green, verified by wasm cargo check). dbus is vendored so clippy --all-features works on runners without libdbus headers.

Consumption: lazy, no extra requests, env precedence

• New CredentialStoreAuthentication combinator wraps the existing env policy: runs it first, returns non-4xx untouched, and only on an auth-relevant 4xx reads the credential blob (once per process, cached via OnceCell; spawn_blocking for the sync keyring call) and issues a forced bearer retry when a stored glob matches.
• Crucially it passes the initial response through: no matching record means the original response is returned with zero extra requests (404 is routine on the resolve path, so a naive retry layer would have doubled round-trips for logged-in users; asserted by request-counting tests).
• Local/offline commands, public unauthenticated reads, and never-logged-in users never touch the keyring. Steady-state keychain reads are silent per platform.
• Precedence is SYSAND_CRED_* > stored login > unauthenticated, end to end.
• After a forced retry from a stored credential still fails with any 4xx and the record's expires_at is past, a hint suggests re-running sysand auth login (any 4xx, since some hosts answer 404 on bad auth); at most once per record per process.

Login validation (always on)

• Discovery-first: the login fetches sysand-index-config.json once, both for glob scoping and to learn api_root. The API surface is probed only when discovery explicitly advertised api_root (a new api_root_advertised flag distinguishes it from the plain-URL runtime default), so static indexes are never phantom-probed.
• Per-surface probe: unauthenticated baseline, then a forced-auth retry only if the baseline was 4xx. A surface counts as tested only when the credential was actually exercised; a public 200 proves nothing. v1/whoami is forced-only. Probes never follow redirects (a redirected probe counts as not tested, naming the target). A 429 is never a verdict (rate-limited probes count as not tested; this was a spec bug found during implementation and fixed in both code and plan).
• Refusal rule: store if any exercised surface accepted (warning about the rest); refuse, before any write, only when at least one exercised surface rejected and none accepted; nothing exercised stores as "stored, not validated". Output always scopes the claim: validated (read), validated (api), validated (read, api), or stored, not validated.
• A rejected login whose read surface challenged with WWW-Authenticate: Basic says the index uses username/password auth and routes to SYSAND_CRED_<X>_BASIC_USER / _BASIC_PASS.
• A successful whoami probe persists subject, token name/prefix, and expires_at into the record for auth status.
• There is no validation opt-out, no flag and no core parameter: offline or unreachable indexes degrade gracefully through the refusal rule (nothing exercised the credential, so it stores as "stored, not validated" with warnings), false refusals are engineered out (429 and redirects are never verdicts; acceptance by any exercised surface wins), and SYSAND_CRED_* remains the escape hatch for pathological middleboxes.
• The login discovery fetch is an unauth baseline with a forced-bearer retry (carrying the just-entered secret) on any 4xx, including 404, distinguishing a hidden discovery document from an absent one. A fully private dynamic index therefore gets its api_root learned, disjoint roots covered, and can validate as validated (read, api); a forced 404 is treated as an authoritative "no document".

Glob scoping

• Derived globs are globset-escaped (globset::escape(<normalized root>) + **), so URLs containing glob metacharacters, including IPv6 literals like https://[::1]:8000/, cannot silently fail to match themselves (tested).
• Coverage is anchored on the user-supplied discovery URL and extended with the resolved index_root / api_root only when not already prefix-covered; disjoint hosts get their own escaped glob. Tests assert the discovery document URL, index.json URL, and upload URL each match the derived set.
• Templated locations, both discovery-advertised index_roots and typed login targets, are anchored at their literal prefix (clamped to at least scheme://authority/; a template with no safe anchor is rejected with a pointer to SYSAND_CRED_*).
• Stored globs are the login-time snapshot and are never auto-updated from live discovery (the security boundary from the plan's trust model).

Publish integration

• Bearer selection is now source precedence: env matches first (single match within env; ambiguity errors per source without falling back), then stored logins, replacing the old flat exactly-one-match rule. Trusted publishing is unchanged and still wins in CI.
• The selected bearer carries provenance, so an upload 401/403 says where the credential came from ("from SYSAND_CRED_<LABEL>" vs "from your stored login for <key>") with matching remediation; a 403 additionally points at sysand auth status (catches a project token scoped to a different project).
• A stored bearer with a clearly past expires_at (60 s skew margin) stops publish before the archive upload; the server's 401 remains the authority.
• Publish reads the keyring only when env has no matching bearer (asserted by panicking-provider tests).

Protocol spec and server counterpart

• design/index-api-protocol.md gains a Token Identity section specifying GET v1/whoami under the resolved api_root: bearer auth, 200 body (subject.type user/project/oidc with per-type subject.name semantics, token.name null for exchanged OIDC tokens, non-secret token.prefix, expires_at with Z suffix), 401 with unspecified body, MUST NOT consume single-use credentials, MAY rate limit.
• The endpoint itself is implemented and tested in the sysand-index repo (routed at api/v1/whoami, reusing the existing bearer resolution across all three token types). It is already deployed on sysand.com, satisfying the ordering requirement that whoami be live before this client (a validating login against an official index without it would refuse valid tokens on a 404).

Testing and verification

• 463 sysand-core tests (with the keyring feature enabled) and 164 sysand CLI tests, all green on the final branch; cargo check -p sysand-core -F networking (keyring off) and wasm cargo check green; prek run -a clean; every commit DCO-signed.
• CLI integration tests never touch the real OS keyring: a debug-build-only SYSAND_TEST_CREDENTIAL_STORE seam selects a file-backed or absent backend (hard-errors in release builds), enabling end-to-end login/status/logout tests including a PTY-driven hidden-prompt test and a no-secret-leak assertion on the no-keyring path.
• Notable properties covered by tests: zero keyring reads for public/unauthenticated flows, exactly one blob read per process, original response returned with no extra request on no-match, S1-S4 validation matrix from the plan, refusal never mutates an existing record, IPv6/template glob derivation, pre-B7 blob compatibility.

Deferred and follow-ups

Deferred by design (plan section 10): auth set / auth unset, basic auth via --username, --pattern, longest-prefix glob tie-breaking, and P2 protocol enforcement (a separate breaking change). Follow-ups noted during implementation: exposing do_auth_* through the py/java bindings, templated login targets, a CI test lane with -F keyring, and cargo-deny verification of the new deps (keyring, fd-lock, believed MIT/Apache-compatible).

docs.sysand.com documentation. The user-facing docs for this feature are written and reviewed (a multi-lens review pass, fixes applied) on the docs/sysand-auth branch in the sysand-index repo, and are live on the staging docs preview. That branch is 16 pages: the client authentication explanation / how-to / reference, the auth login / logout / status / whoami command pages, publish and configuration cross-links, and the index-side API-token pages. It is deliberately gated on this PR shipping: docs.sysand.com deploys from main, and the docs policy forbids documenting functionality that has not been released, so the branch merges to main (and goes live) only once the release carrying this change is out.

---

🤖 Generated with Claude Code

https://claude.ai/code/session_01EfyrkJkeo3mRngVoppJjUN

[consideRatio]

Doing some UX testing, dumping notes here. (Transcripts below refreshed to match the gutter-aligned output that landed after the original capture.)

• sysand auth status output included no coloring, it should look nicer
• sysand auth login <gitlab index with {path}> doesn't work yet, but plan was that it should
• sysand auth whoami felt like a very relevant command that was missing, and there is for example kubectl auth whoami, so adding it makes sense to me
• Fix output alignment to mirror other commands
• It is not so obvious that indexes configured are typically not considered, its mostly the default index that is considered with commands. not sure what i suggest if anything, sysand auth status
• (semi unrelated) sysand publish requires the flag --index which is a bit weird UX wise, while sysand auth login takes an optional positional argument. I think it smells a bit, not sure what i suggest.
• (unrelated) no nice tab indentation etc on sysand info --iri ... output

Testing with staging.sysand.com

$ sysand auth login https://staging.sysand.com
  Logging in to index `https://staging.sysand.com/`
Enter token for `https://staging.sysand.com/`:
error: credential for `https://staging.sysand.com/` was rejected by the index API (`v1/whoami`) and accepted by no surface; nothing was stored

note: pass `-v`/`--verbose` to output additional logs

$ sysand auth login https://staging.sysand.com
  Logging in to index `https://staging.sysand.com/`
Enter token for `https://staging.sysand.com/`:
      Stored credential for `https://staging.sysand.com/` (validated (api))
      Covers https://staging.sysand.com/**

$ sysand auth login https://staging.sysand.com
  Logging in to index `https://staging.sysand.com/`
Enter token for `https://staging.sysand.com/`:
   Replacing existing credential for `https://staging.sysand.com/`
      Stored credential for `https://staging.sysand.com/` (validated (api))
      Covers https://staging.sysand.com/**

$ sysand auth status
      Stored https://staging.sysand.com/  validated (api)
             patterns: https://staging.sysand.com/**
             subject: user admin
             token prefix: sysand_u_53a8fea5
             expires: 2026-10-17 11:27:09 UTC (expires in 89 days)

$ sysand build
    Building kpar `/home/erik/dev/sensmetry/sysand-test-repo/output/proj0-4.0.21.kpar`
updating file metadata (1/1)
   Including readme from `/home/erik/dev/sensmetry/sysand-test-repo/README.md`
   Including license from `/home/erik/dev/sensmetry/sysand-test-repo/LICENSES/MIT.txt`

$ sysand publish --index https://staging.sysand.com
warning: KPAR does not contain a changelog file CHANGELOG.md;
         it is recommended to provide it to inform users of
         the changes between versions
  Publishing admin/proj0 v4.0.21 to https://staging.sysand.com/
   Published new release successfully

$ sysand auth logout https://staging.sysand.com
 Logging out from index `https://staging.sysand.com/`
     Removed stored credential for `https://staging.sysand.com/`

$ sysand auth status
No credentials configured (no stored logins, no `SYSAND_CRED_*` variables).

Testing with private index on GitHub

$ sysand auth login "https://raw.githubusercontent.com/consideratio/priv-index-example-github/refs/heads/index/"
  Logging in to index `https://raw.githubusercontent.com/consideratio/priv-index-example-github/refs/heads/index/`
Enter token for `https://raw.githubusercontent.com/consideratio/priv-index-example-github/refs/heads/index/`:
      Stored credential for `https://raw.githubusercontent.com/consideratio/priv-index-example-github/refs/heads/index/` (validated (read))
      Covers https://raw.githubusercontent.com/consideratio/priv-index-example-github/refs/heads/index/**

$ sysand info --iri "pkg:sysand/example-publisher-1/example-project-1" --index "https://raw.githubusercontent.com/consideratio/priv-index-example-github/refs/heads/index/"
warning: Direct or transitive usages of SysML v2/KerML standard library packages are
         ignored by default. If you want to process them, pass `--include-std` flag
Name: example-project-1
Publisher: Example Publisher 1
Version: 0.0.1
License: MIT
No usages.

Testing with private index on GitLab

$ sysand auth login "https://gitlab.com/api/v4/projects/84113019/repository/files/{path}/raw?ref=index"
  Logging in to index `https://gitlab.com/api/v4/projects/84113019/repository/files/{path}/raw?ref=index`
Enter token for `https://gitlab.com/api/v4/projects/84113019/repository/files/{path}/raw?ref=index`:
      Stored credential for `https://gitlab.com/api/v4/projects/84113019/repository/files/{path}/raw?ref=index` (validated (read))
      Covers https://gitlab.com/api/v4/projects/84113019/repository/files/**

$ sysand auth status
      Stored https://gitlab.com/api/v4/projects/84113019/repository/files/{path}/raw?ref=index  validated (read)
             patterns: https://gitlab.com/api/v4/projects/84113019/repository/files/**

$ sysand info --iri "pkg:sysand/example-publisher-1/example-project-1" --index "https://gitlab.com/api/v4/projects/84113019/repository/files/{path}/raw?ref=index"
warning: Direct or transitive usages of SysML v2/KerML standard library packages are
         ignored by default. If you want to process them, pass `--include-std` flag
Name: example-project-1
Publisher: Example Publisher 1
Version: 0.0.1
License: MIT
No usages.

[andrius-puksta-sensmetry]

Why require api_root to be explicitly advertised to treat it as such?

[consideRatio]

Without doing so, you could end up not knowing if you failed to access an API or there was no API if the discovery/index was public but not the API, as error codes cant be trusted to distinguish that.

However, I dislike the state of things now with this PR, as we havnt forced sysand publish to require api_root to be explicitly declared.

Wdyt? Make sysand publish require it?

An index must have a file serving index root, but not an API, so it would be good to have the discovery doc (or missing of one) to indicate it - for example with an explicit api_root

[consideRatio]

Not reflected at great length, but asked ai to make such requirement #478 preliminary