Skip to content

chore: route Gradle distribution through MASS pull-through cache#11461

Open
jzh0u wants to merge 1 commit into
masterfrom
jenny.zhou/mass-pullthrough-gradle-dist
Open

chore: route Gradle distribution through MASS pull-through cache#11461
jzh0u wants to merge 1 commit into
masterfrom
jenny.zhou/mass-pullthrough-gradle-dist

Conversation

@jzh0u
Copy link
Copy Markdown
Collaborator

@jzh0u jzh0u commented May 26, 2026

Summary

  • Routes Gradle wrapper distribution downloads through the MASS pull-through cache (mass-read.us1.ddbuild.io) instead of directly hitting services.gradle.org
  • Adds MASS_READ_URL CI variable pointing to the prod MASS read endpoint
  • Fixes GRADLE_VERSION to match the version in gradle/wrapper/gradle-wrapper.properties (8.14.5)
  • On cache miss, MASS returns the upstream URL immediately and fills the cache in the background; subsequent requests serve from S3

How it works

A sed in before_script rewrites gradle-wrapper.properties at CI time:

distributionUrl=https\://services.gradle.org/distributions/...
→
distributionUrl=https\://mass-read.us1.ddbuild.io/internal/artifact/services.gradle.org/distributions/...

MASS pull-through is already enabled in prod (mass-pullthrough-enabled feature flag). The services.gradle.org domain is already in the MASS egress allowlist.

Test plan

  • Verify CI pipeline passes with the rewritten distributionUrl
  • Confirm second CI run hits S3 cache (HTTP 307 → S3 presigned URL) rather than upstream

Part of DPCYMGMT-3608.

🤖 Generated with Claude Code

Fix GRADLE_VERSION to match gradle-wrapper.properties (8.14.5).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@dd-octo-sts
Copy link
Copy Markdown
Contributor

dd-octo-sts Bot commented May 26, 2026

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite Status
Startup 🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results

Startup Time

Scenario This PR master Change
insecure-bank / iast 14,089 ms 13,980 ms +0.8%
insecure-bank / tracing 12,887 ms 13,039 ms -1.2%
petclinic / appsec 16,460 ms 16,426 ms +0.2%
petclinic / iast 16,520 ms 16,506 ms +0.1%
petclinic / profiling 16,425 ms 16,464 ms -0.2%
petclinic / tracing 15,593 ms 16,022 ms -2.7%

Commit: 558c9829 · CI Pipeline · Benchmarking Platform UI


Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

@AlexeyKuznetsov-DD AlexeyKuznetsov-DD requested a review from bric3 May 26, 2026 18:40
Copy link
Copy Markdown
Contributor

@AlexeyKuznetsov-DD AlexeyKuznetsov-DD left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, I can see on CI in logs:

Downloading https://mass-read.us1.ddbuild.io/internal/artifact/services.gradle.org/distributions/gradle-8.14.5-bin.zip
.............10%.............20%.............30%.............40%.............50%.............60%..............70%.............80%.............90%.............100%

But let's get review from @bric3 too.

@jzh0u jzh0u marked this pull request as ready for review May 26, 2026 18:43
@jzh0u jzh0u requested a review from a team as a code owner May 26, 2026 18:43
@dd-octo-sts
Copy link
Copy Markdown
Contributor

dd-octo-sts Bot commented May 26, 2026

Hi! 👋 Thanks for your pull request! 🎉

To help us review it, please make sure to:

  • Add at least one type, and one component or instrumentation label to the pull request

If you need help, please check our contributing guidelines.

@dd-octo-sts dd-octo-sts Bot added the tag: ai generated Largely based on code generated by an AI or LLM label May 26, 2026
@datadog-datadog-prod-us1-2

This comment has been minimized.

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 558c98298f

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread .gitlab-ci.yml
# replace maven central part by MAVEN_REPOSITORY_PROXY in .mvn/wrapper/maven-wrapper.properties
- sed -i "s|https://repo.maven.apache.org/maven2/|$MAVEN_REPOSITORY_PROXY|g" .mvn/wrapper/maven-wrapper.properties
# Route Gradle distribution download through MASS pull-through cache
- sed -i "s|distributionUrl=https\\\\://services.gradle.org/|distributionUrl=https\\\\://${MASS_READ_URL#https://}/internal/artifact/services.gradle.org/|" gradle/wrapper/gradle-wrapper.properties
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Fix sed pattern to match escaped Gradle URL

This substitution never matches gradle/wrapper/gradle-wrapper.properties because the search pattern resolves to distributionUrl=https\://services.gradle.org/ with a single backslash before :, while the file contains https\:// (a literal backslash before ://). As a result, CI leaves distributionUrl unchanged and still downloads from services.gradle.org, so the new MASS pull-through routing does not actually take effect.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks legit.

Comment thread .gitlab-ci.yml
Comment on lines +232 to +233
# Route Gradle distribution download through MASS pull-through cache
- sed -i "s|distributionUrl=https\\\\://services.gradle.org/|distributionUrl=https\\\\://${MASS_READ_URL#https://}/internal/artifact/services.gradle.org/|" gradle/wrapper/gradle-wrapper.properties
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor note, I asked Codex to make this change to be a bit more human-readable:

Suggested change
# Route Gradle distribution download through MASS pull-through cache
- sed -i "s|distributionUrl=https\\\\://services.gradle.org/|distributionUrl=https\\\\://${MASS_READ_URL#https://}/internal/artifact/services.gradle.org/|" gradle/wrapper/gradle-wrapper.properties
# Route Gradle distribution download through MASS pull-through cache
- |
mass_read_host="${MASS_READ_URL#https://}"
mass_read_host="${mass_read_host%/}"
sed -i "/^distributionUrl=/ s|services.gradle.org|${mass_read_host}/internal/artifact/services.gradle.org|" gradle/wrapper/gradle-wrapper.properties

WDYT?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the local vars, but I have doubt on the sed expression. I'm reviewing from mobile so…

Copy link
Copy Markdown
Contributor

@bric3 bric3 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pre-approving, but Gitlab is probably "hydrating" gradle from a cache. I believe this might prevent gradle wrapper from fetching from MASS. I didn't look at the commit history so bear with me if I missed something, otherwise I'd say that in order to test this properly there could be a commit adding a line that deletes .gradle before the main job script executes. Then remove that change before adding the PR to the merge queue.

Comment thread .gitlab-ci.yml
DEPENDENCY_CACHE_POLICY: pull
BUILD_CACHE_POLICY: pull
GRADLE_VERSION: "8.14.4" # must match gradle-wrapper.properties
GRADLE_VERSION: "8.14.5" # must match gradle-wrapper.properties
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

praise: Ouch, good find.

@AlexeyKuznetsov-DD
Copy link
Copy Markdown
Contributor

AlexeyKuznetsov-DD commented May 26, 2026

Pre-approving, but Gitlab is probably "hydrating" gradle from a cache. I believe this might prevent gradle wrapper from fetching from MASS. I didn't look at the commit history so bear with me if I missed something, otherwise I'd say that in order to test this properly there could be a commit adding a line that deletes .gradle before the main job script executes. Then remove that change before adding the PR to the merge queue.

@bric3 AFAIK nightly populate caches jobs runs from scratch -> no-caches, so it will be cached by MASS.
Also we have tests that loads old Gradle distros, I hope they will benefit a lot.

@AlexeyKuznetsov-DD AlexeyKuznetsov-DD added tag: no release notes Changes to exclude from release notes comp: tooling Build & Tooling labels May 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp: tooling Build & Tooling tag: ai generated Largely based on code generated by an AI or LLM tag: no release notes Changes to exclude from release notes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants