feat: Add mirror and mirror-token inputs for custom Python distribution sources - #1302
feat: Add mirror and mirror-token inputs for custom Python distribution sources#1302luhenry wants to merge 3 commits into
mirror and mirror-token inputs for custom Python distribution sources#1302Conversation
|
Relates to #1289 |
|
I've verified it works as expected with https://github.com/riseproject-dev/riscv-runner-sample/actions/runs/24014042747/job/70030253499 |
|
@gdams 👋 would you know who would be a good person to have a loot at that? That would greatly unblock riscv64 on github more generally. Thank you very much! |
I'll ask around internally, just to clarify is your question Python specific or more generally about the state of riscv64 native runners? |
|
Merging that PR would enable a feature which would unblock users like using |
|
Hello! Following up on that. Also, please let me know if there is no chance this will get integrated, so that I can focus on workarounds. |
|
Hi @luhenry, Rebase first. It's conflicting, and 1. Scope For downloads, decide by host: mirror host with Knock-on: the "takes precedence" lines in both input descriptions, the auth bullets in 2. Throw-as-control-flow costs 3 retries after rebase. 3. Docs and tests disagree on manifest auth. Docs say 4. Slash branches silently lose auth. 5. 6. PyPy and GraalPy ignore 7. The E2E job doesn't exercise the new paths. Once those are addressed, happy to take another look. Thanks! |
|
I’m OoO this week but will absolutely rebase and address feedback next week! Thank you very much 🙏 |
|
Hi @luhenry, |
…bution sources Users who need custom CPython builds (internal mirrors, GHES-hosted forks, special build configurations, compliance builds, air-gapped runners) could not previously point setup-python at anything other than actions/python-versions. Adds two new inputs: - `mirror`: base URL hosting versions-manifest.json and the Python distributions it references. Defaults to the existing https://raw.githubusercontent.com/actions/python-versions/main. - `mirror-token`: optional token used to authenticate requests to the mirror. If `mirror` is a raw.githubusercontent.com/{owner}/{repo}/{branch} URL, the manifest is fetched via the GitHub REST API (authenticated rate limit applies); otherwise the action falls back to a direct GET of {mirror}/versions-manifest.json. Token interaction ----------------- `token` is never forwarded to arbitrary hosts. Auth resolution is per-URL: 1. if mirror-token is set, use mirror-token 2. else if token is set AND the target host is github.com, *.github.com, or *.githubusercontent.com, use token 3. else send no auth Cases: Default (no inputs set) mirror = default raw.githubusercontent.com URL, mirror-token empty, token = github.token. → manifest API call and tarball downloads use `token`. Identical to prior behavior. Custom raw.githubusercontent.com mirror (e.g. personal fork) mirror-token empty, token = github.token. → manifest API call and tarball downloads use `token` (target hosts are GitHub-owned). Custom non-GitHub mirror, no mirror-token mirror-token empty, token = github.token. → manifest fetched via direct URL (no auth attached), tarball downloads use no auth. `token` is NOT forwarded to the custom host — this is the leak-prevention case. Custom non-GitHub mirror with mirror-token mirror-token set, token may be set. → manifest fetch and tarball downloads use `mirror-token`. Custom GitHub mirror with both tokens set mirror-token wins. Used for both the manifest API call and tarball downloads.
How should I handle that? Are you happy to point to a "external" mirror (can be hosted on github as well)? But then your testing would take a dependency on this external mirror. |
- scope mirror-token to the mirror host and send it verbatim - route non-repo mirrors straight to the URL fetch instead of throwing - authenticate the manifest fetch - warn on slash branches, and on mirror with PyPy/GraalPy - memoize mirror validation - exercise the direct-URL path in the E2E job Addresses actions#1302 (comment) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
Hello @luhenry👋, Thanks for the quick turnaround. A few more things worth a look: 1. The PyPy/GraalPy warning fires on every run. 2. 3. The slash-branch warning misfires on 4. That warning is also wrong about the consequence. It says the manifest is fetched by direct URL without the authenticated rate limit, and the docs caveat calls it "an anonymous direct GET with the 60/hr unauthenticated rate limit", but Since that's the URL the new job uses, fixing (2) will surface this on every run. 5. 6. Scheme-mismatch on the mirror host. Also, the PR description's precedence list ("if |
|
Hello @luhenry 👋 Just a gentle reminder. Whenever you have a moment, could you please address the review points above so we can take another look? Thanks! |
|
@v-priya-kinthali I updated the PR, thanks for your review! :) |
- only warn about PyPy/GraalPy mirror when a custom mirror is set; the
action.yml default made the warning fire on every run
- accept the refs/heads/{branch} raw URL form so it routes via the REST
API instead of tripping the slash-branch warning
- scope mirror-token to the full mirror origin (scheme+host+port) so it
can't leak to a same-host http download_url
- make an invalid mirror fatal on the auth path, matching getManifestUrl
- fix warning/docs that wrongly claimed the raw fallback is anonymous
- force a manifest fetch in the mirror integration job (check-latest) so
it actually contacts the mirror instead of using the preinstalled cache
Description:
Users who need custom CPython builds (internal mirrors, GHES-hosted forks, special build configurations, compliance builds, air-gapped runners) could not previously point setup-python at anything other than actions/python-versions.
Adds two new inputs:
mirror: base URL hosting versions-manifest.json and the Python distributions it references. Defaults to the existing https://raw.githubusercontent.com/actions/python-versions/main.mirror-token: optional token used to authenticate requests to the mirror.If
mirroris ahttps://raw.githubusercontent.com/{owner}/{repo}/{branch}URL, the manifest is fetched via the GitHub REST API (authenticated rate limit applies); otherwise the action falls back to a direct GET of{mirror}/versions-manifest.json. The{branch}may also be given in therefs/heads/{branch}form.This approach is largely inspired from how it's done in
actions/setup-nodeToken interaction
Neither
tokennormirror-tokenis forwarded to a host the user did not nominate. Auth is resolved per request URL:mirror-tokenis set AND the request origin (scheme + host + port) equals themirrororigin, usemirror-token, sent verbatimtokenis set AND the target host isgithub.com,*.github.com, or*.githubusercontent.com, usetoken, sent astoken <token>One exception: when
mirroris araw.githubusercontent.com/{owner}/{repo}/{branch}URL, the manifest is fetched from the GitHub REST API (api.github.com, a different origin than the mirror). Theremirror-tokenis preferred overtoken— with thetokenprefix the API requires — because naming a repo mirror is explicit intent to read that repo.Cases:
Related issue:
Fixes #1288
Check list: