Problem Statement
OpenShell Providers v2 submit provider credentials, such as API keys, through the
normal gateway API. Historically, provider records could contain inline
plaintext credential values in gateway persistence. That is acceptable only as an
upgrade-compatibility read path, not as the long-term storage model for new
provider writes.
Credential storage should be gateway-owned. Users submit provider credentials
through the normal Providers v2 create/update flow. The gateway owner decides
whether those submitted secrets are stored in OpenShell's default encrypted
database credential store or in an external backend such as Kubernetes Secrets or
OpenBao. Provider configuration must not point at arbitrary pre-existing backend
secrets.
The first concrete use cases are:
- Default gateway deployments storing provider API keys in encrypted database
credential objects.
- Kubernetes deployments storing provider API keys in OpenShell-managed
Kubernetes Secrets.
- OpenBao deployments storing provider API keys in OpenShell-managed KV paths.
- Single-player/local environments later using OS-native stores such as macOS
Keychain.
This issue only targets Providers v2. Providers v1 should not be extended.
Proposed Design
Add a Credential Driver interface, modeled after the compute driver boundary,
that lets the gateway store, resolve, and delete provider credentials through a
gateway-owned storage backend.
Credential storage should support:
- Default encrypted database credential storage when no external credential
driver is configured.
- In-tree gateway drivers for common external backends.
- Out-of-process drivers connected over a Unix domain socket.
- At most one enabled external credential driver at a time, matching the current
compute-driver model.
- Legacy inline database credentials as a read-compatible upgrade path for
provider records created before credential storage handles.
- Opaque gateway-managed credential handles persisted in provider records.
The public provider API remains provider.credentials / openshell provider create --credential. provider.credential_handles is internal gateway state and
must be rejected when supplied by clients.
Example provider create flow:
openshell provider create \
--name nvidia-prod \
--type nvidia \
--credential NVIDIA_API_KEY
When no external credential driver is enabled, the gateway stores the submitted
credential in the default encrypted database credential store, clears inline
credential values before persisting the provider record, and stores only opaque
handles in the provider record. Existing provider records that already contain
inline plaintext credentials remain readable for upgrade compatibility.
Example gateway configuration for the default encrypted database credential
store:
[openshell.gateway.credential_storage]
key_encryption_key_path = "/var/lib/openshell/credentials/key-encryption-key.bin"
Or, for multi-replica deployments where all gateway replicas must share the same
key-encryption key:
[openshell.gateway.credential_storage]
key_encryption_key_env = "OPENSHELL_GATEWAY_CREDENTIAL_KEY_ENCRYPTION_KEY"
Example gateway configuration for Kubernetes Secrets:
[openshell.gateway]
credential_drivers = ["kubernetes-secrets"]
[openshell.credential_drivers.kubernetes-secrets]
namespace = "openshell"
Example gateway configuration for OpenBao:
[openshell.gateway]
credential_drivers = ["openbao"]
[openshell.credential_drivers.openbao]
address = "http://openbao.openbao.svc.cluster.local:8200"
mount = "secret"
kv_version = "2"
auth_method = "kubernetes"
role = "openshell-gateway"
External drivers may use the advanced UDS transport:
[openshell.gateway]
credential_drivers = ["enterprise-secrets"]
[openshell.credential_drivers.enterprise-secrets]
transport = "uds"
socket_path = "/run/openshell/credential-driver.sock"
Built-in external drivers default to in-tree transport and do not require a
transport field.
Credential driver contract:
GetCapabilities: report driver identity and feature support.
StoreCredential: store or overwrite one provider credential and return an
opaque handle.
ResolveCredentials: resolve opaque handles into string secret values when
the gateway materializes a sandbox provider environment.
DeleteCredential: delete one stored provider credential handle.
ListCredentials: optional driver-owned inspection surface.
Backend layout is OpenShell-managed:
- The default encrypted database store stores one encrypted credential object per
provider credential outside the provider record. Each object contains an
encrypted value envelope, and the provider record stores only the handle.
- Kubernetes Secrets stores one OpenShell-managed Secret per provider credential
using deterministic names derived from provider name and credential key.
- OpenBao stores one OpenShell-managed KV entry per provider credential under a
managed prefix such as openshell/provider-credentials/.
- Users do not provide backend Secret names, paths, keys, versions, or
namespaces in provider configuration.
Provider delete and credential removal should delete backend credential objects
before completing the provider database write/delete. Cleanup failures fail
closed.
Acceptance Criteria
- Providers v2 credentials can be stored through the active gateway credential
storage path.
- Providers v1 remain unchanged.
- When no external credential driver is configured, new provider credentials are
stored in the default encrypted database credential store.
- Existing provider records with inline plaintext database credentials remain
readable for upgrade compatibility.
- Gateway configuration rejects more than one enabled external credential
driver.
- Gateway configuration rejects explicit
credential_drivers = []; omitting the
field selects default encrypted database credential storage.
- User-supplied
credential_handles are rejected by provider create/update.
- Stored provider records contain opaque handles, not secret values, for new
provider credential writes.
- Provider API responses redact credential values and do not expose internal
handles.
- Secret values are resolved only at runtime and are never logged.
- Provider update overwrites the existing backend handle when possible.
- Provider update of a legacy inline provider preserves untouched inline
credentials while moving newly submitted credential values into credential
storage.
- Provider delete and credential removal delete backend objects and fail closed
on backend cleanup errors.
- The default encrypted database credential store encrypts provider credential
values with gateway-owned key material and can reuse that key material across
gateway restarts.
- The Helm chart creates and retains a shared key-encryption-key Secret for the
default encrypted database store when no external credential driver is enabled.
- The Kubernetes Secrets driver stores, resolves, and deletes provider API keys
using gateway Kubernetes RBAC.
- The OpenBao driver stores, resolves, and deletes provider API keys from a
deployed OpenBao instance.
- Targeted Kubernetes e2e coverage validates Kubernetes Secrets and OpenBao
storage backends one at a time using the local Skaffold/Helm development
environment.
Alternatives Considered
Keeping only inline plaintext credentials was rejected because it forces provider
records to own secret material directly and does not provide defense in depth for
default gateway deployments. Inline credentials remain supported only as a
legacy read path for provider records created before credential handles.
Using provider-authored references to existing backend secrets was rejected after
design review. It expands provider configuration into backend addressing and
policy, while the intended ownership model is that OpenShell internal driver
logic manages backend destinations.
Using environment variables as the primary credential driver was rejected because
environment variables are an input/discovery mechanism, not a durable backend
storage contract. They may remain useful for local provider discovery.
Resolving credentials inside sandbox runtimes was rejected because backend
access should remain controlled by the gateway owner and should not expand
sandbox authority.
Agent Investigation
- Existing compute drivers provide a suitable model for in-tree and
UDS-connected driver implementations.
- Provider credential storage and resolution belong in the gateway because the
gateway owner controls backend access and policy.
- Providers v2 is the right extension point; Providers v1 should remain
unchanged.
- Upgrade compatibility is preserved by reading existing inline database
credentials while routing new writes through credential storage handles.
- Supporting only one enabled external credential driver simplifies routing and
matches the current compute-driver model.
- Default encrypted database credential storage improves the baseline deployment
posture without requiring Kubernetes Secrets or OpenBao.
- Kubernetes Secrets and OpenBao should be implemented first because they cover
the primary production deployment paths.
- OpenBao validation should use the existing Skaffold and Helm development
environment so behavior is tested against a real Kubernetes deployment.
Problem Statement
OpenShell Providers v2 submit provider credentials, such as API keys, through the
normal gateway API. Historically, provider records could contain inline
plaintext credential values in gateway persistence. That is acceptable only as an
upgrade-compatibility read path, not as the long-term storage model for new
provider writes.
Credential storage should be gateway-owned. Users submit provider credentials
through the normal Providers v2 create/update flow. The gateway owner decides
whether those submitted secrets are stored in OpenShell's default encrypted
database credential store or in an external backend such as Kubernetes Secrets or
OpenBao. Provider configuration must not point at arbitrary pre-existing backend
secrets.
The first concrete use cases are:
credential objects.
Kubernetes Secrets.
Keychain.
This issue only targets Providers v2. Providers v1 should not be extended.
Proposed Design
Add a Credential Driver interface, modeled after the compute driver boundary,
that lets the gateway store, resolve, and delete provider credentials through a
gateway-owned storage backend.
Credential storage should support:
driver is configured.
compute-driver model.
provider records created before credential storage handles.
The public provider API remains
provider.credentials/openshell provider create --credential.provider.credential_handlesis internal gateway state andmust be rejected when supplied by clients.
Example provider create flow:
When no external credential driver is enabled, the gateway stores the submitted
credential in the default encrypted database credential store, clears inline
credential values before persisting the provider record, and stores only opaque
handles in the provider record. Existing provider records that already contain
inline plaintext credentials remain readable for upgrade compatibility.
Example gateway configuration for the default encrypted database credential
store:
Or, for multi-replica deployments where all gateway replicas must share the same
key-encryption key:
Example gateway configuration for Kubernetes Secrets:
Example gateway configuration for OpenBao:
External drivers may use the advanced UDS transport:
Built-in external drivers default to in-tree transport and do not require a
transportfield.Credential driver contract:
GetCapabilities: report driver identity and feature support.StoreCredential: store or overwrite one provider credential and return anopaque handle.
ResolveCredentials: resolve opaque handles into string secret values whenthe gateway materializes a sandbox provider environment.
DeleteCredential: delete one stored provider credential handle.ListCredentials: optional driver-owned inspection surface.Backend layout is OpenShell-managed:
provider credential outside the provider record. Each object contains an
encrypted value envelope, and the provider record stores only the handle.
using deterministic names derived from provider name and credential key.
managed prefix such as
openshell/provider-credentials/.namespaces in provider configuration.
Provider delete and credential removal should delete backend credential objects
before completing the provider database write/delete. Cleanup failures fail
closed.
Acceptance Criteria
storage path.
stored in the default encrypted database credential store.
readable for upgrade compatibility.
driver.
credential_drivers = []; omitting thefield selects default encrypted database credential storage.
credential_handlesare rejected by provider create/update.provider credential writes.
handles.
credentials while moving newly submitted credential values into credential
storage.
on backend cleanup errors.
values with gateway-owned key material and can reuse that key material across
gateway restarts.
default encrypted database store when no external credential driver is enabled.
using gateway Kubernetes RBAC.
deployed OpenBao instance.
storage backends one at a time using the local Skaffold/Helm development
environment.
Alternatives Considered
Keeping only inline plaintext credentials was rejected because it forces provider
records to own secret material directly and does not provide defense in depth for
default gateway deployments. Inline credentials remain supported only as a
legacy read path for provider records created before credential handles.
Using provider-authored references to existing backend secrets was rejected after
design review. It expands provider configuration into backend addressing and
policy, while the intended ownership model is that OpenShell internal driver
logic manages backend destinations.
Using environment variables as the primary credential driver was rejected because
environment variables are an input/discovery mechanism, not a durable backend
storage contract. They may remain useful for local provider discovery.
Resolving credentials inside sandbox runtimes was rejected because backend
access should remain controlled by the gateway owner and should not expand
sandbox authority.
Agent Investigation
UDS-connected driver implementations.
gateway owner controls backend access and policy.
unchanged.
credentials while routing new writes through credential storage handles.
matches the current compute-driver model.
posture without requiring Kubernetes Secrets or OpenBao.
the primary production deployment paths.
environment so behavior is tested against a real Kubernetes deployment.