fix: preserve caller-owned Hugging Face cache sidecars by mldangelo-oai · Pull Request #1792 · promptfoo/modelaudit

mldangelo-oai · 2026-07-30T20:57:26Z

Summary

Preserve pre-existing Hugging Face and ONNX sidecar files stored in caller-owned cache directories, including directories whose names begin with modelaudit_hf_.
Replace filename-based cache ownership inference with explicit provenance supplied only for CLI-created temporary cache roots.
Require resolved containment of both the download directory and individual sidecar before deleting owned temporary files; fail closed for symlink escapes and path-resolution errors.
Preserve normal and interrupted cleanup of genuinely invocation-owned temporary downloads and document the fix under [Unreleased].

Validation

Reproduced the customer-owned ONNX sidecar deletion on current main before the fix.
Full Hugging Face source suite: 554 passed, with two explicitly disabled large-artifact tests skipped.
Hugging Face streaming CLI suite: 50 passed; focused ownership, symlink, error, generator-close, and ONNX sidecar regression suite passed.
Independent native reviews additionally ran 1,030 tests, including optional ONNX malicious/benign cases, with no findings.
Ruff lint and formatting plus changed-file Python 3.12 mypy passed.
Three independent exact-head native Codex security reviews and fresh verifier receipt completed without findings.

mldangelo-oai · 2026-07-30T20:57:43Z

@codex review

chatgpt-codex-connector · 2026-07-30T20:59:39Z

Codex Review: Didn't find any major issues. Keep it up!

Reviewed commit: 685f5d7fc9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

github-actions · 2026-07-30T21:00:05Z

Workflow run and artifacts

Performance Benchmarks

Compared 13 shared benchmarks with a regression threshold of 15%.
Status: 0 regressions, 0 improved, 13 stable, 0 new, 0 missing.
Aggregate shared-benchmark median: 4.419s -> 4.352s (-1.5%).

Workload	Benchmark	Target	Size	Files	Baseline	Current	Change	Status
`nested-payload-review`	`tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_nested_payload_review[nested_raw]`	`nested_raw`	78 B	1	289.5us	274.1us	-5.3%	stable
`warm-cache-rescan`	`tests/benchmarks/test_scan_benchmarks.py::test_scan_warm_cached_repository_rescan`	`release-candidate`	547.3 KiB	32	149.91ms	154.26ms	+2.9%	stable
`rejected-basic-auth-candidates`	`tests/benchmarks/test_scan_benchmarks.py::test_rejected_basic_auth_candidates_scan_linearly`	`-`	371.1 KiB	1	2.565s	2.493s	-2.8%	stable
`padded-multi-stream-upload`	`tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_padded_multi_stream_upload`	`multi_stream_padded`	4.1 KiB	1	348.2us	339.2us	-2.6%	stable
`nested-payload-review`	`tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_nested_payload_review[nested_base64]`	`nested_base64`	98 B	1	297.3us	292.4us	-1.6%	stable
`direct-malicious-upload`	`tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_direct_malicious_upload`	`malicious_reduce`	52 B	1	228.3us	224.8us	-1.6%	stable
`single-checkpoint-preflight`	`tests/benchmarks/test_scan_benchmarks.py::test_scan_single_checkpoint_before_load`	`single_checkpoint.pkl`	183.0 KiB	1	104.17ms	105.56ms	+1.3%	stable
`duplicate-heavy-registry`	`tests/benchmarks/test_scan_benchmarks.py::test_scan_duplicate_registry_snapshot`	`registry-snapshot`	915.2 KiB	13	580.67ms	574.42ms	-1.1%	stable
`nested-payload-review`	`tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_nested_payload_review[nested_hex]`	`nested_hex`	130 B	1	302.9us	306.0us	+1.0%	stable
`mixed-model-repository`	`tests/benchmarks/test_scan_benchmarks.py::test_scan_release_candidate_repository`	`release-candidate`	547.3 KiB	32	642.06ms	646.52ms	+0.7%	stable
`chunked-upload-stream`	`tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_chunked_upload_stream`	`chunked_stream`	278.2 KiB	1	114.05ms	114.83ms	+0.7%	stable
`suspicious-pickle-intake`	`tests/benchmarks/test_scan_benchmarks.py::test_scan_suspicious_pickle_intake`	`suspicious-intake`	183.8 KiB	4	152.15ms	151.51ms	-0.4%	stable
`clean-training-checkpoint`	`tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_clean_training_checkpoint`	`safe_large`	278.2 KiB	1	110.20ms	110.65ms	+0.4%	stable

Copilot

Pull request overview

This PR hardens Hugging Face streaming download cleanup to avoid deleting caller-owned cache sidecar files (notably ONNX external data), by replacing directory-name heuristics with explicit provenance and by enforcing resolved-path containment checks before any cleanup.

Changes:

Introduces explicit cache ownership provenance (_cache_dir_owned) for Hugging Face streaming downloads and requires it for sidecar cleanup decisions.
Strengthens cleanup safety by requiring strict path resolution and containment checks (failing closed on resolution/symlink issues).
Adds regression tests covering ownership, symlink escape, resolution-failure behavior, and generator-close cleanup behavior; documents the fix in the changelog.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.

File	Description
`tests/utils/sources/test_huggingface.py`	Adds focused tests ensuring streaming cleanup only removes invocation-owned sidecars and fails closed on symlink/resolution edge cases.
`modelaudit/utils/sources/huggingface.py`	Reworks `_should_cleanup_hf_streaming_context_file` to require explicit ownership and strict containment; threads ownership through streaming flow.
`modelaudit/cli.py`	Marks CLI-created temporary HF cache roots as owned (`_cache_dir_owned`) and resolves the temp path strictly.
`CHANGELOG.md`	Records the behavior fix under `[Unreleased]` bug fixes.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

chatgpt-codex-connector · 2026-07-30T21:03:11Z

Security review completed. No security issues were found in this pull request.

Reviewed commit: 685f5d7fc9

View security finding report

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

You comment "@codex security review"
A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

…-caller-owned-cache-cleanup-20260730

Copilot

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 53101456b7

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

…-1792-author

chatgpt-codex-connector · 2026-08-31T19:14:37Z

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-08-31T22:25:49.794748Z	`ce7db4f`	Manual request
🔒 Security Review	✅ Completed 2026-08-31T22:25:06.162232Z	`ce7db4f`	Manual request

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

mldangelo-oai · 2026-08-31T19:15:21Z

@codex review

mldangelo-oai · 2026-08-31T19:15:28Z

@codex security review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3ba2985611

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-08-31T19:25:19Z

Security review completed. No security issues were found in this pull request.

Reviewed commit: 3ba2985611

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

You comment "@codex security review"
A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

mldangelo-oai · 2026-08-31T20:32:22Z

@codex review

mldangelo-oai · 2026-08-31T20:32:29Z

@codex security review

chatgpt-codex-connector · 2026-08-31T20:39:10Z

Security review completed. No security issues were found in this pull request.

Reviewed commit: d85ecc8648

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

You comment "@codex security review"
A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

chatgpt-codex-connector · 2026-08-31T20:39:18Z

Security review completed. No security issues were found in this pull request.

Reviewed commit: d85ecc8648

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

You comment "@codex security review"
A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d85ecc8648

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

mldangelo-oai · 2026-08-31T21:12:38Z

@codex review

mldangelo-oai · 2026-08-31T21:12:55Z

@codex security review

chatgpt-codex-connector · 2026-08-31T21:16:10Z

Codex Review: Didn't find any major issues. Breezy!

Reviewed commit: 0a0211b90a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-08-31T21:20:56Z

Security review completed. No security issues were found in this pull request.

Reviewed commit: 0a0211b90a

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

You comment "@codex security review"
A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

mldangelo-oai · 2026-08-31T22:20:16Z

@codex review

mldangelo-oai · 2026-08-31T22:20:16Z

@codex security review

chatgpt-codex-connector · 2026-08-31T22:25:04Z

Security review completed. No security issues were found in this pull request.

Reviewed commit: ce7db4fab7

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

You comment "@codex security review"
A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

chatgpt-codex-connector · 2026-08-31T22:25:48Z

Codex Review: Didn't find any major issues. Can't wait for the next one!

Reviewed commit: ce7db4fab7

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-08-31T22:26:29Z

Security review completed. No security issues were found in this pull request.

Reviewed commit: ce7db4fab7

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

You comment "@codex security review"
A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

fix: preserve caller-owned Hugging Face cache sidecars

685f5d7

Copilot AI review requested due to automatic review settings July 30, 2026 20:57

Copilot started reviewing on behalf of mldangelo-oai July 30, 2026 20:57 View session

Copilot AI reviewed Jul 30, 2026

View reviewed changes

mldangelo mentioned this pull request Aug 1, 2026

fix(joblib): fail closed on inconclusive warning scans #1796

Open

mldangelo enabled auto-merge (squash) August 1, 2026 13:31

mldangelo mentioned this pull request Aug 1, 2026

fix(picklescan): diagnose Windows call-graph source-stability failures #1789

Merged

Merge remote-tracking branch 'origin/main' into mdangelo/codex/fix-hf…

5310145

…-caller-owned-cache-cleanup-20260730

Copilot AI review requested due to automatic review settings August 1, 2026 17:15

Copilot started reviewing on behalf of mldangelo August 1, 2026 17:16 View session

Copilot AI reviewed Aug 1, 2026

View reviewed changes

chatgpt-codex-connector Bot reviewed Aug 1, 2026

View reviewed changes

Comment thread modelaudit/utils/sources/huggingface.py Outdated

mldangelo-oai added 6 commits August 31, 2026 12:49

Merge commit 'e061182355fed862d4abafe5baf110382937a393' into codex/pr…

6a38496

…-1792-author

fix: preserve caller-owned Hugging Face cache sidecars

ea89d43

fix: clean invocation-owned Hugging Face sidecars

a101b12

fix: isolate Hugging Face streaming cleanup

7c25d0d

fix: bound Hugging Face streaming staging

60b478c

fix: quarantine Hugging Face staging cleanup

3ba2985

chatgpt-codex-connector Bot reviewed Aug 31, 2026

View reviewed changes

Comment thread modelaudit/utils/sources/huggingface.py

Comment thread modelaudit/cli.py

mldangelo-oai added 2 commits August 31, 2026 19:57

fix: preserve streamed findings on cleanup failure

ae8d0e0

test: stage mocked Hugging Face streams

d85ecc8

chatgpt-codex-connector Bot reviewed Aug 31, 2026

View reviewed changes

Comment thread modelaudit/utils/sources/huggingface.py Outdated

Comment thread modelaudit/cli.py Outdated

fix: align Hugging Face streaming staging roots

0a0211b

test: stage selected Hugging Face filename mock

ce7db4f

Uh oh!

Conversation

mldangelo-oai commented Jul 30, 2026

Summary

Validation

Uh oh!

mldangelo-oai commented Jul 30, 2026

Uh oh!

chatgpt-codex-connector Bot commented Jul 30, 2026

Uh oh!

github-actions Bot commented Jul 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Performance Benchmarks

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

chatgpt-codex-connector Bot commented Jul 30, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

chatgpt-codex-connector Bot commented Aug 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codex Review Summary

Uh oh!

mldangelo-oai commented Aug 31, 2026

Uh oh!

mldangelo-oai commented Aug 31, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

chatgpt-codex-connector Bot commented Aug 31, 2026

Uh oh!

mldangelo-oai commented Aug 31, 2026

Uh oh!

mldangelo-oai commented Aug 31, 2026

Uh oh!

chatgpt-codex-connector Bot commented Aug 31, 2026

Uh oh!

chatgpt-codex-connector Bot commented Aug 31, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

mldangelo-oai commented Aug 31, 2026

Uh oh!

mldangelo-oai commented Aug 31, 2026

Uh oh!

chatgpt-codex-connector Bot commented Aug 31, 2026

Uh oh!

chatgpt-codex-connector Bot commented Aug 31, 2026

Uh oh!

mldangelo-oai commented Aug 31, 2026

Uh oh!

mldangelo-oai commented Aug 31, 2026

Uh oh!

chatgpt-codex-connector Bot commented Aug 31, 2026

Uh oh!

chatgpt-codex-connector Bot commented Aug 31, 2026

Uh oh!

chatgpt-codex-connector Bot commented Aug 31, 2026

Uh oh!

Reviewers

Assignees

github-actions Bot commented Jul 30, 2026 •

edited

Loading

chatgpt-codex-connector Bot commented Aug 31, 2026 •

edited

Loading