Use a Deployment instead of a StatefulSet for the controller

[antonipp]

Description

While trying out the agent-sandbox project, I noticed that the main controller is deployed as a StatefulSet. I couldn't really find a justification for it in the commit history, so this looks like an oversight. The controller is stateless and doesn't require a stable network identity, so I don't really see a reason to use a much heavier abstraction here. Moreover, STS ordered rollout semantics are unnecessary for a stateless controller and could complicate scaling/updates if leader election is enabled for HA.

So my PR switches the controller to be deployed as a simple Deployment, I think it will make operations easier for everyone

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: antonipp
Once this PR has been reviewed and has the lgtm label, please assign justinsb for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[netlify]

✅ Deploy Preview for agent-sandbox canceled.

Name	Link
🔨 Latest commit	1571613
🔍 Latest deploy log	https://app.netlify.com/projects/agent-sandbox/deploys/69302cc49364080008d430ef

[k8s-ci-robot]

Hi @antonipp. Thanks for your PR.

I'm waiting for a github.com member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[janetkuo]

/ok-to-test

[janetkuo]

To provide some more context, the original discussion regarding StatefulSet vs. Deployment is here: #13 (comment)

Note that if we decide to accept this change, this requires an additional migration step when upgrading Agent Sandbox to this version.

[antonipp]

ah thanks, I didn't see this conversation. But tbh I still don't really understand why an STS would be better for leader election specifically? I don't really see why the overhead would be worth it. (cc @justinsb)

Also FWIW, we've been running dozens of different controllers over the years and from what I've seen they've all been Deployments. When it comes to the identity, they seem to simply be using os.Hostname(). We don't need static STS identities at all.

Random examples:

• cluster-autoscaler https://github.com/kubernetes/autoscaler/blob/fb2899a594ed99489add5c46e3680a460776154b/cluster-autoscaler/main.go#L403-L449 / https://github.com/kubernetes/autoscaler/blob/fb2899a594ed99489add5c46e3680a460776154b/cluster-autoscaler/charts/cluster-autoscaler/templates/deployment.yaml#L4
• Cilium Operator https://github.com/cilium/cilium/blob/f8a57c416f0aac506e5725f939fe42b8464885fe/operator/cmd/root.go#L469-L538 / https://github.com/cilium/cilium/blob/cb94478fa365a6331ff0a79ee82c223bb6d4b936/install/kubernetes/cilium/templates/cilium-operator/deployment.yaml#L4
• CSI Snapshotter: https://github.com/kubernetes-csi/external-snapshotter/blob/2c37c1c04f835acb2f22e9d543e1f68bcef4cb30/cmd/snapshot-controller/main.go#L343-L371 / https://github.com/kubernetes-csi/external-snapshotter/blob/2c37c1c04f835acb2f22e9d543e1f68bcef4cb30/deploy/kubernetes/snapshot-controller/setup-snapshot-controller.yaml#L9

They just have way less constraints and are easier to operate.

[barney-s]

This is an acceptable change. We can switch back to sts later if needed. Would you check the docs and check if they need a change.

[barney-s]

Thanks for the changes

[barney-s]

While removing serviceName is correct for a Deployment, it's worth double-checking if any other part of the system implicitly relied on the stable pod names generated by the StatefulSet (e.g., in logging, monitoring, or custom scripts).

[antonipp]

ok actually found 1 reference in examples/jupyterlab/README.md but that's it (and it's good that nothing depended on this logic)

[barney-s]

While changing the kind to Deployment is the correct move for a stateless controller, this is a breaking change for existing users. An upgrade will not be smooth as the new Deployment won't manage the old StatefulSet's pods.

Please add a note to the RELEASE.md or a dedicated upgrade guide explaining the manual steps required, which likely involve deleting the old StatefulSet before applying the new manifests.

[antonipp]

I am aware it's a breaking change but not really sure what's the actual process for documenting it should be. RELEASE.md feels wrong and an upgrade guide is overkill for such as simple change (especially since the project is in the early days and users should expect the internals to change)

Maybe something we just call out in the Github release notes somehow?

[antonipp]

/retest

[k8s-ci-robot]

@antonipp: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
presubmit-agent-sandbox-e2e-test	1571613	link	true	/test presubmit-agent-sandbox-e2e-test

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.