fix(joblib): fail closed on inconclusive warning scans

[mldangelo-oai]

Summary

Keep Joblib scans failed closed whenever NumPy-wrapper validation is inconclusive, regardless of what the embedded pickle scanner reports.

The defect

JoblibScanner.scan() failed an inconclusive scan only when there were no WARNING/CRITICAL findings:

if scan_outcome == INCONCLUSIVE and not has_security_findings and not has_trusted_incomplete_tail:
    result.finish(success=False)
else:
    result.finish(success=not result.has_errors)

That conflates "we found something" with "we finished the analysis". When findings exist the scanner takes the else branch and reports success=True as long as no finding is CRITICAL — even though scan_outcome=inconclusive and analysis_incomplete=true.

Joblib was the outlier. Roughly a dozen scanners already use the unconditional form this PR adopts — lightgbm, catboost, coreml, rknn, r_serialized, paddle, xgboost, skops, flax_msgpack, llamafile, pytorch_zip, torchserve_mar all finish with scan_outcome != INCONCLUSIVE and not has_errors. This change brings joblib in line, keeping only the joblib-specific trusted_incomplete_tail exemption.

Correction to the original justification

This PR was originally filed as the fix for a recurring Windows CI failure of test_scan_fails_closed_on_protocol1_pickle_in_numpy_wrapper_tail. That claim was wrong and has been removed.

The Windows lane passes on #1780, #1783, #1788, #1790 and #1791 — every one of which contains main without this fix. The test is therefore intermittent, not deterministically failing, and this change is not what makes it pass. The Windows failures on #1792/#1793 were separately traced to stale July-30 runs that predated #1795.

The defect above is real and reproducible on POSIX independently of any platform flake, which is the basis for merging this. It is a correctness fix, not a CI fix.

Why it stays hidden on POSIX in normal runs: ScanResult.finish() computes success and self._merged_children_success, and the embedded pickle scanner usually returns success=False, which pins the parent to False. The joblib-level guard only becomes observable when the embedded result reports success.

Coverage

Two regressions, both failing on main and passing here:

• test_scan_fails_closed_on_invalid_numpy_wrapper_with_origin_warning — stubs the embedded scanner to isolate the guard.
• test_scan_fails_closed_when_embedded_pickle_reports_success — keeps the real pickle scanner, real payload parsing, and real metadata composition, forcing only the embedded verdict.

Demonstrated behaviour change with the embedded verdict forced to success:

	success	scan_outcome	trusted_incomplete_tail
main	True	inconclusive	absent
this PR	False	inconclusive	absent

Validation

• tests/scanners/test_joblib_scanner_codecs.py, test_joblib_scanner.py, test_pickle_scanner.py: 472 passed.
• Ruff lint + format clean; mypy clean on both changed files.

[mldangelo-oai]

@codex review

[Copilot]

Pull request overview

This PR fixes a fail-open behavior in the Joblib scanner: when Joblib NumPy-wrapper validation is inconclusive, the scan now fails closed even if the embedded pickle analysis reports warning-level findings (e.g., origin verification warnings). This aligns the scanner’s success semantics with the project’s fail-closed expectations for incomplete analysis and adds a deterministic regression for a Windows-observed failure mode.

Changes:

• Update JoblibScanner.scan() to always return success=False for scan_outcome=inconclusive results unless a trusted incomplete tail is explicitly present.
• Add a regression test covering an invalid NumPy wrapper combined with an embedded warning-level finding, ensuring the scan fails closed and is not cacheable.
• Document the fix in the root changelog under Bug Fixes.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File	Description
modelaudit/scanners/joblib_scanner.py	Makes inconclusive Joblib scans fail closed regardless of embedded warning findings (unless a trusted incomplete tail is present).
tests/scanners/test_joblib_scanner_codecs.py	Adds a deterministic regression reproducing the prior fail-open path (inconclusive wrapper + embedded WARNING finding).
CHANGELOG.md	Notes the Joblib fail-closed behavior change under Bug Fixes.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[github-actions]

Workflow run and artifacts

Performance Benchmarks

Compared 13 shared benchmarks with a regression threshold of 15%.
Status: 0 regressions, 0 improved, 13 stable, 0 new, 0 missing.
Aggregate shared-benchmark median: 4.012s -> 4.000s (-0.3%).

Workload	Benchmark	Target	Size	Files	Baseline	Current	Change	Status
nested-payload-review	tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_nested_payload_review[nested_raw]	nested_raw	78 B	1	246.6us	240.7us	-2.4%	stable
direct-malicious-upload	tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_direct_malicious_upload	malicious_reduce	52 B	1	195.6us	198.6us	+1.5%	stable
padded-multi-stream-upload	tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_padded_multi_stream_upload	multi_stream_padded	4.1 KiB	1	305.6us	301.4us	-1.4%	stable
suspicious-pickle-intake	tests/benchmarks/test_scan_benchmarks.py::test_scan_suspicious_pickle_intake	suspicious-intake	183.8 KiB	4	146.71ms	148.56ms	+1.3%	stable
rejected-basic-auth-candidates	tests/benchmarks/test_scan_benchmarks.py::test_rejected_basic_auth_candidates_scan_linearly	-	371.1 KiB	1	2.226s	2.207s	-0.9%	stable
chunked-upload-stream	tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_chunked_upload_stream	chunked_stream	278.2 KiB	1	122.08ms	123.04ms	+0.8%	stable
mixed-model-repository	tests/benchmarks/test_scan_benchmarks.py::test_scan_release_candidate_repository	release-candidate	547.3 KiB	32	595.21ms	599.44ms	+0.7%	stable
nested-payload-review	tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_nested_payload_review[nested_base64]	nested_base64	98 B	1	266.6us	265.4us	-0.4%	stable
single-checkpoint-preflight	tests/benchmarks/test_scan_benchmarks.py::test_scan_single_checkpoint_before_load	single_checkpoint.pkl	183.0 KiB	1	97.41ms	97.70ms	+0.3%	stable
duplicate-heavy-registry	tests/benchmarks/test_scan_benchmarks.py::test_scan_duplicate_registry_snapshot	registry-snapshot	915.2 KiB	13	574.93ms	574.00ms	-0.2%	stable
clean-training-checkpoint	tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_clean_training_checkpoint	safe_large	278.2 KiB	1	120.35ms	120.16ms	-0.2%	stable
nested-payload-review	tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_nested_payload_review[nested_hex]	nested_hex	130 B	1	274.6us	274.7us	+0.0%	stable
warm-cache-rescan	tests/benchmarks/test_scan_benchmarks.py::test_scan_warm_cached_repository_rescan	release-candidate	547.3 KiB	32	128.12ms	128.12ms	-0.0%	stable

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 713940829d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Security review completed. No security issues were found in this pull request.

Reviewed commit: 713940829d

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

• You comment "@codex security review"
• A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

Suppressed comments (1)

tests/scanners/test_joblib_scanner_codecs.py:1426

• The test forces an embedded pickle scan to report success=True by directly mutating ScanResult.success and the private _merged_children_success field after finish(). This bypasses ScanResult.finish() invariants (e.g., restored-critical handling) and couples the test to private implementation details. Prefer the public trust_merged_child_failures() API and re-run finish(success=True) to recompute success consistently.

        if not result.success:
            result.success = True
            result._merged_children_success = True

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-09-01T04:48:54.266889Z	5e5909e	Manual request
🔒 Security Review	✅ Completed 2026-09-01T04:50:50.618882Z	5e5909e	Manual request

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[mldangelo-oai]

@codex review

[mldangelo-oai]

@codex security review

[chatgpt-codex-connector]

Codex Review: Didn't find any major issues. More of your lovely PRs please.

Reviewed commit: 5e5909e85a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Security review completed. No security issues were found in this pull request.

Reviewed commit: 5e5909e85a

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

• You comment "@codex security review"
• A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.