feat: add rubygems dependent-counts closure variant (CM-1241)#4329
Merged
Conversation
Signed-off-by: Uroš Marolt <uros@marolt.me>
Contributor
There was a problem hiding this comment.
Pull request overview
Adds RubyGems reverse dependent-count ingestion using the existing exact closure pipeline.
Changes:
- Adds the RubyGems SQL closure and workflow variant.
- Registers the job across bootstrap, CLI, monitoring, and DAL types.
- Documents the 200 GB billing ceiling.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
ingestJobs.ts |
Registers the RubyGems job kind. |
triggerBootstrap.ts |
Adds CLI support. |
monitorOsspckgs.ts |
Adds monitoring mapping. |
ingestDependentCounts.ts |
Configures the RubyGems workflow variant. |
bootstrapOsspckgs.ts |
Runs RubyGems during bootstrap. |
README.md |
Documents the job and byte ceiling. |
dependentCountsSql.ts |
Builds the RubyGems reverse closure query. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Signed-off-by: Uroš Marolt <uros@marolt.me>
Signed-off-by: Uroš Marolt <uros@marolt.me>
Signed-off-by: Uroš Marolt <uros@marolt.me>
Signed-off-by: Uroš Marolt <uros@marolt.me>
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit b9ada52. Configure here.
… (CM-1241) Signed-off-by: Uroš Marolt <uros@marolt.me>
mbani01
previously approved these changes
Jul 13, 2026
…1241) Signed-off-by: Uroš Marolt <uros@marolt.me>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
RubyGems packages never got dependent-count fields filled.
Dependentsreverseindex (source for the existing edge-based path) only covers NPM/MAVEN/PYPI/CARGO
— verified on BQ, no RUBYGEMS rows. Adds a 4th closure variant (mirrors GO/NUGET)
that computes the exact reverse transitive closure from
RubyGemsRequirementsLatest.Changes
rubygemsvariant iningestDependentCountsworkflow (own job kind, staging table, guard baseline)buildRubygemsDependentCountsSqlindependentCountsSql.ts, reusing the shared closure scriptbootstrapOsspckgs,triggerBootstrap,monitorOsspckgsdependent_counts_rubygemsadded toOsspckgsJobKindType of change
JIRA ticket
https://linuxfoundation.atlassian.net/browse/CM-1241
Note
Medium Risk
Touches weekly BQ script ingest and merges into
packagespopularity fields; behavior matches proven GO/NUGET patterns but adds new BigQuery spend and operational surface.Overview
RubyGems packages can now get
dependent_count,transitive_dependent_count, anddependent_repos_countvia a fourth ingest path, mirroring GO/NUGET. deps.dev’sDependentsindex still excludes RubyGems, so counts come from an exact reverse transitive closure BQ script overRubyGemsRequirementsLatest(RuntimeDependenciesonly), reusing the sharedbuildClosureScript/isScriptexport path.Wiring adds job kind
dependent_counts_rubygems(staging table, 200 GBmaximumBytesBilledcap), loops it throughbootstrapOsspckgs,triggerBootstrap, andmonitorOsspckgs, and documents it in ADR-0004 and the packages_worker README.The row-count guard now keys Slack guidance off job kind (
RubyGemsRequirementsLatestvsDependents, and no misleading “snapshot date” hint for*Latestmanifest kinds).Reviewed by Cursor Bugbot for commit 6b94574. Bugbot is set up for automated code reviews on this repo. Configure here.