fix(cells) Add rollout option and support for apigateway deployments#118098
fix(cells) Add rollout option and support for apigateway deployments#118098markstory wants to merge 1 commit into
Conversation
Add a new rollout option and logic to use the api_gateway deployments in all cells. I've defaulted the US to 1.0 as that region has already had a rollout done for api_gateway deployments. Refs INC-2198
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit c22f330. Configure here.
| register( | ||
| "hybridcloud.rpc.use_pooling_rate", | ||
| type=Float, | ||
| default=1.0, |
There was a problem hiding this comment.
RPC pooling option accidentally deleted
High Severity
Replacing apigateway.proxy.use_gateway_address removed the register call for hybridcloud.rpc.use_pooling_rate, but _get_connection in hybridcloud/rpc/service.py still calls in_random_rollout for that key. options.get then raises UnknownOption on cross-silo RPC traffic.
Reviewed by Cursor Bugbot for commit c22f330. Configure here.
| register( | ||
| "apigateway.proxy.use_gateway_address", | ||
| type=Float, | ||
| default=0.0, | ||
| flags=FLAG_AUTOMATOR_MODIFIABLE, | ||
| ) | ||
| register( | ||
| "hybridcloud.rpc.use_pooling_rate", | ||
| type=Float, | ||
| default=1.0, | ||
| "apigateway.proxy.cell-rollout", | ||
| type=Dict, | ||
| default={ | ||
| "us": 1.0, | ||
| "de": 0.0, | ||
| "us2": 0.0, | ||
| "s4s2": 0.0, | ||
| }, | ||
| flags=FLAG_AUTOMATOR_MODIFIABLE, | ||
| ) |
There was a problem hiding this comment.
Bug: The removed option hybridcloud.rpc.use_pooling_rate is still in use, causing in_random_rollout to raise an unhandled UnknownOption error on every RPC call.
Severity: CRITICAL
Suggested Fix
Re-add the registration for the hybridcloud.rpc.use_pooling_rate option in src/sentry/hybridcloud/options.py. If this option is no longer needed, remove the corresponding call to in_random_rollout("hybridcloud.rpc.use_pooling_rate") from src/sentry/hybridcloud/rpc/service.py.
Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.
Location: src/sentry/hybridcloud/options.py#L202-L214
Potential issue: The option `hybridcloud.rpc.use_pooling_rate` was removed from the
options registry in `src/sentry/hybridcloud/options.py`, but it is still referenced in
`src/sentry/hybridcloud/rpc/service.py` through a call to `in_random_rollout`. This
function internally calls `options.get()` without a `silent` parameter, which raises an
`UnknownOption` exception for unregistered options. Because this exception is not
handled in the call stack, every remote RPC call will crash, causing a major failure in
inter-silo communication.
Also affects:
src/sentry/hybridcloud/rpc/service.py:527
Did we get this right? 👍 / 👎 to inform future reviews.


Add a new rollout option and logic to use the api_gateway deployments in all cells. I've defaulted the US to 1.0 as that region has already had a rollout done for api_gateway deployments.
Refs INC-2198