Skip to content

(feat): compile-time and runtime structural mutation detection#35

Merged
mgyoo86 merged 5 commits intomasterfrom
feat/detect_resize
Mar 14, 2026
Merged

(feat): compile-time and runtime structural mutation detection#35
mgyoo86 merged 5 commits intomasterfrom
feat/detect_resize

Conversation

@mgyoo86
Copy link
Member

@mgyoo86 mgyoo86 commented Mar 14, 2026

Summary

  • Add compile-time detection (PoolMutationError) that catches resize!/push!/pop!/deleteat!/empty!/splice!/sizehint! on pool-backed arrays inside @with_pool blocks — zero runtime cost
  • Add runtime detection (@warn maxlog=1) that checks wrapper Memory/DataRef identity and length at rewind! time — catches mutations that escape compile-time analysis
  • Full support across CPU, CUDA, and Metal backends

Why warnings, not errors?

The pool is self-healing: acquire! unconditionally resets the wrapper's memory reference to the backing vector. Structural mutation (e.g. resize!) does NOT cause data corruption or undefined behavior. The only consequences are:

  • Pooling benefits (zero-alloc reuse) may be lost
  • Temporary extra memory retention (GPU: VRAM) until next acquire at the same slot

Runtime warning with maxlog=1 is appropriate — advisory, one-time per session, non-blocking.
Compile-time remains a hard error as a zero-cost lint for definite anti-patterns.

Detection mechanisms

Layer What How Cost
Compile-time resize!, push!, pop!, etc. in @with_pool AST walking at macro expansion Zero runtime
Runtime Check 1 Memory/DataRef reallocation Identity comparison at rewind ~5ns/slot
Runtime Check 2 Wrapper length > backing length Length comparison at rewind ~5ns/slot

mgyoo86 added 3 commits March 14, 2026 09:33
Add PoolMutationError that catches resize!, push!, pop!, append!, and
other structural mutation calls on pool-backed variables at macro
expansion time (zero runtime cost). Pool-backed arrays share memory
with the pool's backing storage, so structural mutations corrupt pool
invariants and cause undefined behavior on rewind!.

Reuses existing _extract_acquired_vars infrastructure from escape
detection. Wired into all 5 macro entry points (block + function
forms, CPU + backend variants).
Detect resize!/push!/append! on pool-backed wrappers at rewind time by
comparing wrapper state against backing vector (zero extra storage).

CPU: MemoryRef identity + length divergence (1.11+ only, gated)
CUDA: DataRef identity + length divergence
Metal: DataRef identity + length divergence

Warns (not throws) to avoid skipping cleanup of remaining TypedPools.
Runtime mutation detection (resize!/push! on pool-backed arrays) now emits
a one-time @warn instead of alarming language. The pool self-heals on next
acquire!, so mutation is not data corruption — only pooling benefits may be
lost with temporary extra memory retention.

- Update warning messages across CPU, CUDA, and Metal backends
- Compile-time PoolMutationError: keep hard error, soften docstring tone
- Add GPU runtime mutation test suites (CUDA + Metal)
- Add mutation_safety_review.md for external review context
@codecov
Copy link

codecov bot commented Mar 14, 2026

Codecov Report

❌ Patch coverage is 96.83544% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 96.55%. Comparing base (2d870d3) to head (9031328).
⚠️ Report is 4 commits behind head on master.

Files with missing lines Patch % Lines
src/debug.jl 93.47% 3 Missing ⚠️
src/macros.jl 98.14% 2 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##           master      #35      +/-   ##
==========================================
+ Coverage   96.53%   96.55%   +0.01%     
==========================================
  Files          14       14              
  Lines        2568     2726     +158     
==========================================
+ Hits         2479     2632     +153     
- Misses         89       94       +5     
Files with missing lines Coverage Δ
src/AdaptiveArrayPools.jl 100.00% <ø> (ø)
src/state.jl 97.10% <100.00%> (+0.04%) ⬆️
src/macros.jl 97.98% <98.14%> (+0.01%) ⬆️
src/debug.jl 94.93% <93.47%> (-0.40%) ⬇️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds structural-mutation detection for pool-backed arrays, combining a compile-time macro check (hard error) with a rewind-time runtime check (one-time warning) across CPU, CUDA, and Metal backends.

Changes:

  • Add compile-time AST-based detection of structural mutations inside @with_pool / @maybe_with_pool blocks via PoolMutationError.
  • Add runtime detection at rewind! time that warns if cached wrappers’ underlying memory/data refs or lengths diverge from backing storage.
  • Add CPU/CUDA/Metal test coverage for both compile-time and runtime mutation detection, and wire new tests into each test runner.

Reviewed changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated 9 comments.

Show a summary per file
File Description
src/macros.jl Calls _check_structural_mutation during macro codegen and introduces PoolMutationError/MutationPoint plus AST-walking helpers.
src/debug.jl Implements CPU runtime _check_wrapper_mutation! (Julia ≥ 1.11) and BitArray variant; adds fallback no-op for unsupported pools.
src/state.jl Hooks runtime mutation checking into _invalidate_released_slots! before poisoning/invalidation.
ext/AdaptiveArrayPoolsCUDAExt/debug.jl Adds CUDA runtime mutation detection (_check_wrapper_mutation!) and calls it during slot invalidation.
ext/AdaptiveArrayPoolsMetalExt/debug.jl Adds Metal runtime mutation detection (_check_wrapper_mutation!) and calls it during slot invalidation.
src/AdaptiveArrayPools.jl Exports PoolMutationError and MutationPoint.
test/test_compile_mutation.jl New tests for compile-time mutation detection helpers and macro integration.
test/test_runtime_mutation.jl New CPU runtime warning tests for wrapper mutation detection.
test/runtests.jl Includes new CPU compile/runtime mutation test files.
test/cuda/test_runtime_mutation.jl New CUDA runtime warning tests.
test/cuda/runtests.jl Includes CUDA runtime mutation tests.
test/metal/test_runtime_mutation.jl New Metal runtime warning tests.
test/metal/runtests.jl Includes Metal runtime mutation tests.
claudedocs/mutation_safety_review.md Adds design/safety analysis notes documenting the rationale and mechanisms.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

mgyoo86 added 2 commits March 14, 2026 12:34
Copilot review feedback: copy() with no args is invalid Julia, and
the copy suggestion is unnecessary since mutation is not harmful.
Simplify all warning/error messages to just recommend acquire!(pool, T, n).
Change "Request the exact size" → "Consider requesting the exact size
if known in advance" and "Fix:" → "Tip:" for a gentler advisory tone.
@mgyoo86 mgyoo86 merged commit 5bf6990 into master Mar 14, 2026
12 checks passed
@mgyoo86 mgyoo86 deleted the feat/detect_resize branch March 14, 2026 19:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants