diff --git a/docs/src/basics/safety-rules.md b/docs/src/basics/safety-rules.md index 64674c74..2f7439d1 100644 --- a/docs/src/basics/safety-rules.md +++ b/docs/src/basics/safety-rules.md @@ -105,20 +105,20 @@ end end ``` -## Debugging with POOL_DEBUG +## Debugging with RUNTIME_CHECK -Enable runtime safety checks during development: +Enable runtime safety checks during development by setting the `runtime_check` preference: -```julia -using AdaptiveArrayPools -AdaptiveArrayPools.POOL_DEBUG[] = true - -@with_pool pool function test() - v = acquire!(pool, Float64, 100) - return v # Will warn about returning pool-backed array -end +```toml +# LocalPreferences.toml +[AdaptiveArrayPools] +runtime_check = 1 # 0 = off (default), 1 = on ``` +**Restart Julia** after changing this setting. When enabled, returning a pool-backed array from a `@with_pool` block throws a `PoolRuntimeEscapeError` with the exact source location. + +See [Safety](../features/safety.md) for full details on what `RUNTIME_CHECK = 1` enables (poisoning, structural invalidation, escape detection, borrow tracking). + ## acquire! vs unsafe_acquire! | Function | Returns | Best For | diff --git a/docs/src/features/configuration.md b/docs/src/features/configuration.md index f4f26ccd..fa9114cc 100644 --- a/docs/src/features/configuration.md +++ b/docs/src/features/configuration.md @@ -4,10 +4,13 @@ AdaptiveArrayPools can be configured via `LocalPreferences.toml`: ```toml [AdaptiveArrayPools] -use_pooling = false # ⭐ Primary: Disable pooling entirely -cache_ways = 8 # Advanced: N-way cache size (default: 4) +use_pooling = false # ⭐ Primary: Disable pooling entirely +runtime_check = 1 # Safety: Enable runtime safety checks +cache_ways = 8 # Advanced: N-way cache size (default: 4) ``` +All compile-time preferences require **restarting Julia** to take effect. + ## Compile-time: STATIC_POOLING (⭐ Primary) **The most important configuration.** Completely disable pooling to make `acquire!` behave like standard allocation. @@ -50,26 +53,40 @@ Use `pooling_enabled(pool)` to check if pooling is active. All pooling code is **completely eliminated at compile time** (zero overhead). -## Runtime: MAYBE_POOLING +## Compile-time: RUNTIME_CHECK -Only affects `@maybe_with_pool`. Toggle without restart. +Enable runtime safety checks to catch pool-escape bugs. See [Safety](safety.md) for full details. + +```toml +# LocalPreferences.toml +[AdaptiveArrayPools] +runtime_check = 1 # enable (0 = off, 1 = on) +# runtime_check = true # also accepted +``` + +Or programmatically: ```julia -MAYBE_POOLING[] = false # Disable -MAYBE_POOLING[] = true # Enable (default) +using Preferences +Preferences.set_preferences!(AdaptiveArrayPools, "runtime_check" => 1) +# Restart Julia for changes to take effect ``` -## Runtime: POOL_DEBUG +Accepts both `Bool` and `Int` values — internally normalized to `Int`: +- `false` / `0` → off (zero overhead, all safety branches eliminated) +- `true` / `1` → on (poisoning + invalidation + escape detection + borrow tracking) + +The safety level is baked into the pool type parameter: `AdaptiveArrayPool{0}` or `AdaptiveArrayPool{1}`. This enables dead-code elimination — at `RUNTIME_CHECK = 0`, all safety branches are completely removed by the compiler. + +## Runtime: MAYBE_POOLING -Enable safety validation to catch direct returns of pool-backed arrays. +Only affects `@maybe_with_pool`. Toggle without restart. ```julia -POOL_DEBUG[] = true # Enable safety checks (development) -POOL_DEBUG[] = false # Disable (default, production) +MAYBE_POOLING[] = false # Disable +MAYBE_POOLING[] = true # Enable (default) ``` -When enabled, returning a pool-backed array from a `@with_pool` block will throw an error. - ## Compile-time: CACHE_WAYS (Julia 1.10 / CUDA only) Configure the N-way cache size for `unsafe_acquire!`. **On Julia 1.11+ CPU, this setting has no effect** — the `setfield!`-based wrapper reuse supports unlimited dimension patterns with zero allocation. @@ -99,6 +116,6 @@ set_cache_ways!(8) | Setting | Scope | Restart? | Priority | Affects | |---------|-------|----------|----------|---------| | `use_pooling` | Compile-time | Yes | ⭐ Primary | All macros, `acquire!` behavior | +| `runtime_check` | Compile-time | Yes | Safety | Poisoning, invalidation, escape detection | | `cache_ways` | Compile-time | Yes | Advanced | `unsafe_acquire!` N-D caching (Julia 1.10 / CUDA only) | | `MAYBE_POOLING` | Runtime | No | Optional | `@maybe_with_pool` only | -| `POOL_DEBUG` | Runtime | No | Debug | Safety validation | diff --git a/docs/src/features/multi-threading.md b/docs/src/features/multi-threading.md index 41350198..0017138e 100644 --- a/docs/src/features/multi-threading.md +++ b/docs/src/features/multi-threading.md @@ -270,7 +270,7 @@ If you encounter unexpected behavior: 1. **Check pool placement**: Is `@with_pool` inside or outside `@threads`? 2. **Check pool sharing**: Is the same pool variable accessed from multiple Tasks? -3. **Enable POOL_DEBUG**: `POOL_DEBUG[] = true` catches some (not all) misuse patterns +3. **Enable RUNTIME_CHECK**: Set `runtime_check = 1` in `LocalPreferences.toml` (restart required) to catch escape bugs --- @@ -281,4 +281,4 @@ If you encounter unexpected behavior: - `@threads` creates one Task per thread → pools are reused within the block - **Always place `@with_pool` inside `@threads`**, not outside - Thread-local pools are **not an alternative** due to stack discipline requirements -- Correct usage is the user's responsibility (no runtime checks for performance) +- Correct usage is the user's responsibility (enable `RUNTIME_CHECK` during development to catch bugs) diff --git a/docs/src/features/safety.md b/docs/src/features/safety.md index 72b387b3..f2399dc8 100644 --- a/docs/src/features/safety.md +++ b/docs/src/features/safety.md @@ -1,6 +1,6 @@ # Pool Safety -AdaptiveArrayPools catches pool-escape bugs at **two levels**: compile-time (macro analysis) and runtime (configurable safety levels). +AdaptiveArrayPools catches pool-escape bugs at **two levels**: compile-time (macro analysis) and runtime (configurable via `RUNTIME_CHECK`). ## Compile-Time Detection @@ -65,67 +65,103 @@ end end ``` -## Runtime Safety Levels +## Runtime Safety (`RUNTIME_CHECK`) For bugs the compiler can't catch (e.g., values hidden behind opaque function calls), runtime safety provides configurable protection via the type parameter `S` in `AdaptiveArrayPool{S}`. -### Level Overview +### Binary System -| Level | Name | CPU | CUDA | Overhead | -|-------|------|-----|------|----------| -| **0** | off | No-op (all branches dead-code-eliminated) | Same | Zero | -| **1** | guard | `resize!(v,0)` + `setfield!` invalidation | NaN/sentinel poisoning + cache clear | ~5ns/slot | -| **2** | full | Level 1 + data poisoning + escape detection at scope exit | Level 1 + device-pointer overlap check | Moderate | -| **3** | debug | Level 2 + acquire call-site tracking | Same | Moderate+ | +| `RUNTIME_CHECK` | State | What Happens | Overhead | +|:-:|-------|--------------|----------| +| **0** | off | All safety branches dead-code-eliminated | **Zero** | +| **1** | on | Poisoning + structural invalidation + escape detection + borrow tracking | ~5ns/slot | -### Why CPU and CUDA Differ at Level 1 +`RUNTIME_CHECK` is a **compile-time constant** — not a runtime toggle. At `RUNTIME_CHECK = 0`, the JIT eliminates all safety branches completely. No `Ref` reads, no conditional branches, no overhead whatsoever. -Both achieve the same goal — **make stale references fail loudly** — but use different mechanisms: +### Enabling Runtime Safety -| | CPU | CUDA | -|---|-----|------| -| **Strategy** | Structural invalidation | Data poisoning | -| **Mechanism** | `resize!(v, 0)` shrinks backing vector to length 0; `setfield!(:size, (0,))` zeroes the array dimensions | `CUDA.fill!(v, NaN)` / `typemax` / `true` fills backing CuVector with sentinel values | -| **Stale access result** | `BoundsError` (array has length 0) | Reads `NaN` or `typemax` (obviously wrong data) | -| **Why not the other way?** | CPU `resize!` is cheap (~0 cost) | CUDA `resize!` calls `CUDA.Mem.free()` — destroys the pooled VRAM allocation | -| **Cache invalidation** | View length/dims zeroed | N-way view cache entries cleared to `nothing` | +Set the `runtime_check` preference in `LocalPreferences.toml` and **restart Julia**: + +```toml +# LocalPreferences.toml +[AdaptiveArrayPools] +runtime_check = 1 # enable all safety checks +# runtime_check = true # also accepted (normalized to 1 internally) +``` -### Setting the Level +Or programmatically: ```julia -using AdaptiveArrayPools +using Preferences +Preferences.set_preferences!(AdaptiveArrayPools, "runtime_check" => 1) +# Restart Julia for changes to take effect +``` -# Enable full safety on CPU + all GPU devices (preserves cached arrays, zero-copy) -set_safety_level!(2) +!!! warning "Restart Required" + `RUNTIME_CHECK` is baked into the pool type at compile time (`AdaptiveArrayPool{S}`). Changing the preference **requires restarting Julia** — it cannot be toggled at runtime. -# Back to zero overhead everywhere -set_safety_level!(0) -``` +### What `RUNTIME_CHECK = 1` Enables -The pool type parameter `S` is a compile-time constant. At `S=0`, the JIT eliminates all safety branches via dead-code elimination — true zero overhead with no `Ref` reads or conditional branches. +When safety is on, `@with_pool` scope exit triggers the following protections: -### Data Poisoning (Level 2+, CPU) +#### 1. Data Poisoning -At Level 1, CPU relies on **structural invalidation** (`resize!` + `setfield!`) which makes stale views throw `BoundsError`. At Level 2+, CPU additionally **poisons** the backing vector data with sentinel values (`NaN`, `typemax`, all-`true` for `BitVector`) *before* structural invalidation. This catches stale access through `unsafe_acquire!` wrappers on Julia 1.10 where `setfield!` on Array is unavailable. +Released arrays are filled with detectable sentinel values **before** structural invalidation: -CUDA already poisons at Level 1 (its primary invalidation strategy), so no additional poisoning step is needed at Level 2. +| Element Type | Poison Value | Detection | +|-------------|-------------|-----------| +| `Float64`, `Float32`, `Float16` | `NaN` | `isnan(x)` returns `true` | +| `Int64`, `Int32`, etc. | `typemax(T)` | Obviously wrong value | +| `ComplexF64`, `ComplexF32` | `NaN + NaN*im` | `isnan(real(x))` | +| `Bool` | `true` | All-true is suspicious | +| Other types | `zero(T)` | Generic fallback | + +#### 2. Structural Invalidation + +After poisoning, stale references are made to fail loudly: + +| | CPU | CUDA | +|---|-----|------| +| **Mechanism** | `resize!(v, 0)` shrinks backing vector; `setfield!(:size, (0,))` zeroes array dimensions | `_resize_to_fit!(v, 0)` shrinks logical length (GPU memory preserved) | +| **Stale access** | `BoundsError` (array has length 0) | `BoundsError` (logical length 0); poisoned data visible on re-acquire | +| **arr_wrapper** | Dimensions set to `(0,)` / `(0,0)` | Same | +| **Why different?** | CPU `resize!` is cheap (~0 cost) | CUDA `resize!` would call `CUDA.Mem.free()` — destroys pooled VRAM | -### Escape Detection (Level 2+) +#### 3. Escape Detection At every `@with_pool` scope exit, the return value is inspected for overlap with pool-backed memory. Recursively checks `Tuple`, `NamedTuple`, `Dict`, `Pair`, `Set`, and `AbstractArray` elements. -Level 3 additionally records each `acquire!` call-site, so the error message pinpoints the exact source line and expression that allocated the escaping array. +```julia +# Throws PoolRuntimeEscapeError at scope exit +@with_pool pool begin + v = acquire!(pool, Float64, 100) + opaque_function(v) # returns v through opaque call +end +``` -### Legacy: `POOL_DEBUG` +#### 4. Borrow Tracking -`POOL_DEBUG[] = true` triggers Level 2 escape detection regardless of `S`. For new code, prefer `set_safety_level!(2)`. +Each `acquire!` call-site is recorded, so escape error messages pinpoint the exact source line and expression that allocated the escaping array: + +``` +PoolEscapeError (runtime, RUNTIME_CHECK >= 1) + + SubArray{Float64, 1, ...} + ← backed by Float64 pool memory, will be reclaimed at scope exit + ← acquired at src/solver.jl:42 + v = acquire!(pool, Float64, n) + + Fix: Wrap with collect() to return an owned copy, or compute a scalar result. +``` ## Recommended Workflow -```julia -# Development / Testing: catch bugs early -set_safety_level!(2) # or 3 for call-site info in error messages +```toml +# Development / Testing (LocalPreferences.toml): +[AdaptiveArrayPools] +runtime_check = 1 # catch bugs early — restart Julia after changing -# Production: zero overhead -set_safety_level!(0) # all safety branches eliminated by the compiler +# Production: +[AdaptiveArrayPools] +runtime_check = 0 # zero overhead — all safety branches eliminated ``` diff --git a/docs/src/reference/api.md b/docs/src/reference/api.md index 0840e575..9e9bab82 100644 --- a/docs/src/reference/api.md +++ b/docs/src/reference/api.md @@ -51,7 +51,7 @@ Default element type is `Float64` (CPU) or `Float32` (CUDA). |--------|-------------| | `STATIC_POOLING` | Compile-time constant to disable all pooling. (alias: `USE_POOLING`) | | `MAYBE_POOLING` | Runtime `Ref{Bool}` for `@maybe_with_pool`. (alias: `MAYBE_POOLING_ENABLED`) | -| `POOL_DEBUG` | Runtime `Ref{Bool}` to enable safety validation. | +| `RUNTIME_CHECK` | Compile-time `Int` constant (0=off, 1=on). Set via `runtime_check` preference. Restart required. | | `set_cache_ways!(n)` | Set N-way cache size (Julia 1.10 / CUDA only; no effect on Julia 1.11+ CPU). | --- diff --git a/ext/AdaptiveArrayPoolsCUDAExt/debug.jl b/ext/AdaptiveArrayPoolsCUDAExt/debug.jl index 5c6c0405..2d33928f 100644 --- a/ext/AdaptiveArrayPoolsCUDAExt/debug.jl +++ b/ext/AdaptiveArrayPoolsCUDAExt/debug.jl @@ -3,12 +3,9 @@ # ============================================================================== # CUDA-specific safety implementations for CuAdaptiveArrayPool{S}. # -# Safety levels on CUDA differ from CPU: -# - Level 0: Zero overhead (all branches dead-code-eliminated) -# - Level 1: Poisoning (NaN/sentinel fill) + structural invalidation via -# _resize_to_fit!(vec, 0) + arr_wrappers invalidation (setfield!(:dims, zeros)) -# - Level 2: Poisoning + escape detection (_validate_pool_return for CuArrays) -# - Level 3: Full + borrow call-site registry + debug messages +# Binary safety system (S=0 off, S=1 all checks): +# - S=0: Zero overhead (all branches dead-code-eliminated) +# - S=1: Poisoning + structural invalidation + escape detection + borrow tracking # # Key difference: CPU uses resize!(v, 0) at Level 1 to invalidate stale SubArrays. # On CUDA, resize!(CuVector, 0) would free GPU memory, so we use @@ -16,15 +13,14 @@ # the GPU allocation (maxsize). Poisoning fills sentinel data before the shrink. # arr_wrappers are invalidated by setting wrapper dims to zeros (matches CPU pattern). -using AdaptiveArrayPools: _safety_level, _validate_pool_return, +using AdaptiveArrayPools: _runtime_check, _validate_pool_return, _set_pending_callsite!, _maybe_record_borrow!, _invalidate_released_slots!, _zero_dims_tuple, _throw_pool_escape_error, - POOL_DEBUG, POOL_SAFETY_LV, PoolRuntimeEscapeError # ============================================================================== -# Poisoning: Fill released CuVectors with sentinel values (Level 1+) +# Poisoning: Fill released CuVectors with sentinel values (S=1) # ============================================================================== _cuda_poison_value(::Type{T}) where {T <: AbstractFloat} = T(NaN) @@ -45,12 +41,12 @@ Fill a CuVector with a detectable sentinel value (NaN for floats, typemax for in end # ============================================================================== -# _invalidate_released_slots! for CuTypedPool (Level 1+) +# _invalidate_released_slots! for CuTypedPool (S=1) # ============================================================================== # # Overrides the no-op fallback in base. On CUDA: -# - Level 0: no-op (base _rewind_typed_pool! gates with S >= 1, so never called) -# - Level 1+: poison released CuVectors + invalidate arr_wrappers +# - S=0: no-op (base _rewind_typed_pool! gates with S >= 1, so never called) +# - S=1: poison released CuVectors + invalidate arr_wrappers # - NO resize!(cuv, 0) — would free GPU memory; use _resize_to_fit! instead @noinline function AdaptiveArrayPools._invalidate_released_slots!( @@ -79,22 +75,22 @@ end end # ============================================================================== -# Borrow Tracking: Call-site recording (Level 3) +# Borrow Tracking: Call-site recording (S=1) # ============================================================================== # # Overrides the no-op AbstractArrayPool fallbacks. # The macro injects pool._pending_callsite = "file:line\nexpr" before acquire calls. # These functions flush that pending info into the borrow log. -"""Record pending callsite for borrow tracking (compiles to no-op when S < 3).""" +"""Record pending callsite for borrow tracking (compiles to no-op when S=0).""" @inline function AdaptiveArrayPools._set_pending_callsite!(pool::CuAdaptiveArrayPool{S}, msg::String) where {S} - S >= 3 && isempty(pool._pending_callsite) && (pool._pending_callsite = msg) + S >= 1 && isempty(pool._pending_callsite) && (pool._pending_callsite = msg) return nothing end -"""Flush pending callsite into borrow log (compiles to no-op when S < 3).""" +"""Flush pending callsite into borrow log (compiles to no-op when S=0).""" @inline function AdaptiveArrayPools._maybe_record_borrow!(pool::CuAdaptiveArrayPool{S}, tp::AbstractTypedPool) where {S} - S >= 3 && _cuda_record_borrow_from_pending!(pool, tp) + S >= 1 && _cuda_record_borrow_from_pending!(pool, tp) return nothing end @@ -118,14 +114,14 @@ end end # ============================================================================== -# Escape Detection: _validate_pool_return for CuArrays (Level 2+) +# Escape Detection: _validate_pool_return for CuArrays (S=1) # ============================================================================== # # CuArray views share the same device buffer, so device pointer overlap # detection works correctly. pointer(::CuArray) returns CuPtr{T}. function AdaptiveArrayPools._validate_pool_return(val, pool::CuAdaptiveArrayPool{S}) where {S} - (S >= 2 || POOL_DEBUG[]) || return nothing + S >= 1 || return nothing _validate_cuda_return(val, pool) return nothing end diff --git a/ext/AdaptiveArrayPoolsCUDAExt/macros.jl b/ext/AdaptiveArrayPoolsCUDAExt/macros.jl index 7acf12f9..bc45dbae 100644 --- a/ext/AdaptiveArrayPoolsCUDAExt/macros.jl +++ b/ext/AdaptiveArrayPoolsCUDAExt/macros.jl @@ -16,12 +16,12 @@ Uses Val dispatch for compile-time resolution and full inlining. @inline AdaptiveArrayPools._get_pool_for_backend(::Val{:cuda}) = get_task_local_cuda_pool() # ============================================================================== -# Pool Type Registration for Closureless Union Splitting +# Pool Type Registration for Compile-Time Type Assertion # ============================================================================== # # `_pool_type_for_backend` is called at macro expansion time to determine the -# concrete pool type for closureless `let`/`if isa` chain generation. -# This enables `@with_pool :cuda` to generate `if _raw isa CuAdaptiveArrayPool{0} ...` -# instead of closure-based `_dispatch_pool_scope`. +# concrete pool type for direct type assertion in macro-generated code. +# This enables `@with_pool :cuda` to generate `pool::CuAdaptiveArrayPool{S}` +# where S is determined by the compile-time const `RUNTIME_CHECK`. AdaptiveArrayPools._pool_type_for_backend(::Val{:cuda}) = CuAdaptiveArrayPool diff --git a/ext/AdaptiveArrayPoolsCUDAExt/state.jl b/ext/AdaptiveArrayPoolsCUDAExt/state.jl index be5b5dae..bba450d0 100644 --- a/ext/AdaptiveArrayPoolsCUDAExt/state.jl +++ b/ext/AdaptiveArrayPoolsCUDAExt/state.jl @@ -115,7 +115,7 @@ end # Type-specific rewind (single type) @inline function AdaptiveArrayPools.rewind!(pool::CuAdaptiveArrayPool{S}, ::Type{T}) where {S, T} if pool._current_depth == 1 - reset!(AdaptiveArrayPools.get_typed_pool!(pool, T)) + reset!(AdaptiveArrayPools.get_typed_pool!(pool, T), S) return nothing end _rewind_typed_pool!(AdaptiveArrayPools.get_typed_pool!(pool, T), pool._current_depth, S) @@ -136,7 +136,7 @@ end end end rewind_exprs = [:(_rewind_typed_pool!(AdaptiveArrayPools.get_typed_pool!(pool, types[$i]), pool._current_depth, S)) for i in reverse(unique_indices)] - reset_exprs = [:(reset!(AdaptiveArrayPools.get_typed_pool!(pool, types[$i]))) for i in unique_indices] + reset_exprs = [:(reset!(AdaptiveArrayPools.get_typed_pool!(pool, types[$i]), S)) for i in unique_indices] return quote if pool._current_depth == 1 $(reset_exprs...) @@ -265,15 +265,15 @@ end # reset! for CuAdaptiveArrayPool # ============================================================================== -function AdaptiveArrayPools.reset!(pool::CuAdaptiveArrayPool) +function AdaptiveArrayPools.reset!(pool::CuAdaptiveArrayPool{S}) where {S} # Fixed slots AdaptiveArrayPools.foreach_fixed_slot(pool) do tp - reset!(tp) + reset!(tp, S) end # Others for tp in values(pool.others) - reset!(tp) + reset!(tp, S) end # Reset depth and bitmask sentinel state @@ -292,8 +292,8 @@ function AdaptiveArrayPools.reset!(pool::CuAdaptiveArrayPool) end # Type-specific reset -@inline function AdaptiveArrayPools.reset!(pool::CuAdaptiveArrayPool, ::Type{T}) where {T} - reset!(AdaptiveArrayPools.get_typed_pool!(pool, T)) +@inline function AdaptiveArrayPools.reset!(pool::CuAdaptiveArrayPool{S}, ::Type{T}) where {S, T} + reset!(AdaptiveArrayPools.get_typed_pool!(pool, T), S) return pool end diff --git a/ext/AdaptiveArrayPoolsCUDAExt/task_local_pool.jl b/ext/AdaptiveArrayPoolsCUDAExt/task_local_pool.jl index 4ab507dd..7b11c230 100644 --- a/ext/AdaptiveArrayPoolsCUDAExt/task_local_pool.jl +++ b/ext/AdaptiveArrayPoolsCUDAExt/task_local_pool.jl @@ -2,7 +2,7 @@ # Task-Local CUDA Pool (Multi-Device Aware) # ============================================================================== # Each Task gets one pool per GPU device to prevent cross-device memory access. -# Pools are parameterized by safety level S (CuAdaptiveArrayPool{S}). +# Pools are parameterized by S (0=off, 1=checks on) via CuAdaptiveArrayPool{S}. const _CU_POOL_KEY = :ADAPTIVE_ARRAY_POOL_CUDA @@ -19,7 +19,7 @@ a dictionary of pools (one per device) in task-local storage, ensuring that: ## Implementation Uses `Dict{Int, CuAdaptiveArrayPool}` in task-local storage, keyed by device ID. -Values are `CuAdaptiveArrayPool{S}` — use `_dispatch_pool_scope` for union splitting. +Values are `CuAdaptiveArrayPool{S}` where S is determined by `RUNTIME_CHECK`. """ @inline function AdaptiveArrayPools.get_task_local_cuda_pool() # 1. Get or create the pools dictionary @@ -35,7 +35,7 @@ Values are `CuAdaptiveArrayPool{S}` — use `_dispatch_pool_scope` for union spl # 3. Get or create pool for this device pool = get(pools, dev_id, nothing) if pool === nothing - pool = CuAdaptiveArrayPool() # Constructor uses POOL_SAFETY_LV[] + pool = CuAdaptiveArrayPool() # Uses RUNTIME_CHECK for initial S pools[dev_id] = pool end @@ -56,32 +56,3 @@ Useful for diagnostics or bulk operations across all devices. end return pools end - -# ============================================================================== -# Safety Level Hook (called from set_safety_level! in base) -# ============================================================================== - -function AdaptiveArrayPools._set_cuda_safety_level_hook!(level::Int) - pools = get(task_local_storage(), _CU_POOL_KEY, nothing) - pools === nothing && return nothing - - # Check that no pool is inside an active scope - for (dev_id, old_pool) in pools - old = old_pool::CuAdaptiveArrayPool - depth = old._current_depth - depth != 1 && throw( - ArgumentError( - "set_safety_level! cannot be called inside an active @with_pool :cuda scope " * - "(device=$dev_id, depth=$depth)" - ) - ) - end - - # Replace all pools (collect keys to avoid mutating Dict during iteration) - for dev_id in collect(keys(pools)) - old = pools[dev_id]::CuAdaptiveArrayPool - pools[dev_id] = _make_cuda_pool(level, old) - end - - return nothing -end diff --git a/ext/AdaptiveArrayPoolsCUDAExt/types.jl b/ext/AdaptiveArrayPoolsCUDAExt/types.jl index b4f03b6c..b76dd2bb 100644 --- a/ext/AdaptiveArrayPoolsCUDAExt/types.jl +++ b/ext/AdaptiveArrayPoolsCUDAExt/types.jl @@ -77,14 +77,11 @@ const GPU_FIXED_SLOT_FIELDS = ( """ CuAdaptiveArrayPool{S} <: AbstractArrayPool -Multi-type GPU memory pool, parameterized by safety level `S` (0–3). +Multi-type GPU memory pool, parameterized by runtime check level `S` (binary: 0 or 1). -## Safety Levels (CUDA-specific) +## Runtime Check Levels - `S=0`: Zero overhead — all safety branches eliminated by dead-code elimination -- `S=1`: Guard — poisoning (NaN/sentinel fill on released vectors) + cache invalidation - (CUDA equivalent of CPU's resize! structural invalidation) -- `S=2`: Full — poisoning + escape detection (`_validate_pool_return`) -- `S=3`: Debug — full + borrow call-site registry + debug messages +- `S=1`: Full checks — poisoning + structural invalidation + escape detection + borrow tracking ## Device Safety Each pool is bound to a specific GPU device. Using a pool on the wrong device @@ -142,52 +139,52 @@ function CuAdaptiveArrayPool{S}() where {S} CUDA.deviceid(dev), "", # _pending_callsite "", # _pending_return_site - nothing # _borrow_log: lazily created at S >= 3 + nothing # _borrow_log: lazily created when S >= 1 ) end -"""Create pool at the current `POOL_SAFETY_LV[]` level.""" -CuAdaptiveArrayPool() = _make_cuda_pool(AdaptiveArrayPools.POOL_SAFETY_LV[]) +"""Create pool with the default `RUNTIME_CHECK` level.""" +CuAdaptiveArrayPool() = CuAdaptiveArrayPool{AdaptiveArrayPools.RUNTIME_CHECK}() # ============================================================================== -# Safety Level Dispatch +# Runtime Check Dispatch # ============================================================================== """ - _safety_level(pool::CuAdaptiveArrayPool{S}) -> Int + _runtime_check(pool::CuAdaptiveArrayPool) -> Bool -Return compile-time constant safety level for CUDA pools. +Return compile-time constant indicating whether runtime safety checks are enabled. +`S >= 1` enables checks; `S == 0` disables (dead-code-eliminated). """ -@inline AdaptiveArrayPools._safety_level(::CuAdaptiveArrayPool{S}) where {S} = S +@inline AdaptiveArrayPools._runtime_check(::CuAdaptiveArrayPool{0}) = false +@inline AdaptiveArrayPools._runtime_check(::CuAdaptiveArrayPool) = true # S >= 1 """ - _make_cuda_pool(s::Int) -> CuAdaptiveArrayPool{s} + _make_cuda_pool(level) -> CuAdaptiveArrayPool -Function barrier: converts runtime `Int` to concrete `CuAdaptiveArrayPool{S}`. -Levels outside 0-3 are clamped (≤0 → 0, ≥3 → 3). +Function barrier: converts runtime check level to concrete `CuAdaptiveArrayPool{S}`. +Accepts `Bool` (`true`→1, `false`→0) or `Int` (used directly as S). """ -@noinline function _make_cuda_pool(s::Int) - s <= 0 && return CuAdaptiveArrayPool{0}() - s == 1 && return CuAdaptiveArrayPool{1}() - s == 2 && return CuAdaptiveArrayPool{2}() - return CuAdaptiveArrayPool{3}() +_make_cuda_pool(runtime_check::Bool) = _make_cuda_pool(Int(runtime_check)) +@noinline function _make_cuda_pool(S::Int) + S == 0 && return CuAdaptiveArrayPool{0}() + return CuAdaptiveArrayPool{1}() end """ - _make_cuda_pool(s::Int, old::CuAdaptiveArrayPool) -> CuAdaptiveArrayPool{s} + _make_cuda_pool(level, old::CuAdaptiveArrayPool) -> CuAdaptiveArrayPool -Create a new pool at safety level `s`, transferring cached arrays and scope state -from `old`. Only reference copies — no memory allocation for underlying GPU buffers. +Create a new CUDA pool, transferring cached arrays and scope state from `old`. +Only reference copies — no memory allocation for underlying GPU buffers. Transferred: all CuTypedPool slots, `others`, depth & touch tracking, device_id. Reset: `_pending_callsite/return_site` (transient macro state), - `_borrow_log` (created fresh when `s >= 3`). + `_borrow_log` (created fresh when S >= 1). """ -@noinline function _make_cuda_pool(s::Int, old::CuAdaptiveArrayPool) - s <= 0 && return _transfer_cuda_pool(Val(0), old) - s == 1 && return _transfer_cuda_pool(Val(1), old) - s == 2 && return _transfer_cuda_pool(Val(2), old) - return _transfer_cuda_pool(Val(3), old) +_make_cuda_pool(runtime_check::Bool, old::CuAdaptiveArrayPool) = _make_cuda_pool(Int(runtime_check), old) +@noinline function _make_cuda_pool(level::Int, old::CuAdaptiveArrayPool) + level == 0 && return _transfer_cuda_pool(Val(0), old) + return _transfer_cuda_pool(Val(1), old) end """Transfer cached arrays and scope state from `old` pool into a new `CuAdaptiveArrayPool{V}`.""" @@ -203,14 +200,12 @@ function _transfer_cuda_pool(::Val{V}, old::CuAdaptiveArrayPool) where {V} old.device_id, "", # _pending_callsite: reset "", # _pending_return_site: reset - V >= 3 ? IdDict{Any, String}() : nothing # _borrow_log + V >= 1 ? IdDict{Any, String}() : nothing # _borrow_log ) end -"""Human-readable safety level label.""" -function _cuda_safety_label(s::Int) +"""Human-readable runtime check label.""" +function _cuda_check_label(s::Int) s <= 0 && return "off" - s == 1 && return "guard" - s == 2 && return "full" - return "debug" + return "on" end diff --git a/ext/AdaptiveArrayPoolsCUDAExt/utils.jl b/ext/AdaptiveArrayPoolsCUDAExt/utils.jl index 18d40512..b4da03f5 100644 --- a/ext/AdaptiveArrayPoolsCUDAExt/utils.jl +++ b/ext/AdaptiveArrayPoolsCUDAExt/utils.jl @@ -58,13 +58,13 @@ end Print statistics for a CUDA adaptive array pool. """ function AdaptiveArrayPools.pool_stats(pool::CuAdaptiveArrayPool{S}; io::IO = stdout) where {S} - # Header with device info and safety level + # Header with device info and runtime check level printstyled(io, "CuAdaptiveArrayPool", bold = true, color = :green) printstyled(io, "{$S}", color = :yellow) printstyled(io, " (device ", color = :dark_gray) printstyled(io, pool.device_id, color = :blue) - printstyled(io, ", safety=", color = :dark_gray) - printstyled(io, _cuda_safety_label(S), color = :yellow) + printstyled(io, ", check=", color = :dark_gray) + printstyled(io, _cuda_check_label(S), color = :yellow) printstyled(io, ")\n", color = :dark_gray) has_content = false @@ -134,7 +134,7 @@ function Base.show(io::IO, pool::CuAdaptiveArrayPool{S}) where {S} total_active[] += tp.n_active end - return print(io, "CuAdaptiveArrayPool{$S}(safety=$(_cuda_safety_label(S)), device=$(pool.device_id), types=$(n_types[]), slots=$(total_vectors[]), active=$(total_active[]))") + return print(io, "CuAdaptiveArrayPool{$S}(check=$(_cuda_check_label(S)), device=$(pool.device_id), types=$(n_types[]), slots=$(total_vectors[]), active=$(total_active[]))") end # Multi-line show diff --git a/src/AdaptiveArrayPools.jl b/src/AdaptiveArrayPools.jl index 087c6e87..575f4cc4 100644 --- a/src/AdaptiveArrayPools.jl +++ b/src/AdaptiveArrayPools.jl @@ -9,10 +9,8 @@ export zeros!, ones!, trues!, falses!, similar!, reshape!, default_eltype # Con export unsafe_zeros!, unsafe_ones!, unsafe_similar! # Unsafe convenience functions export Bit # Sentinel type for BitArray (use with acquire!, trues!, falses!) export @with_pool, @maybe_with_pool -export STATIC_POOLING, MAYBE_POOLING, POOL_DEBUG, POOL_SAFETY_LV, STATIC_POOL_CHECKS -export DEFAULT_SAFETY_LV, set_safety_level! +export STATIC_POOLING, MAYBE_POOLING, RUNTIME_CHECK export PoolEscapeError, EscapePoint -export USE_POOLING, MAYBE_POOLING_ENABLED # Deprecated aliases (backward compat) export checkpoint!, rewind!, reset! export get_task_local_cuda_pool, get_task_local_cuda_pools # CUDA (stubs, overridden by extension) diff --git a/src/bitarray.jl b/src/bitarray.jl index 387f35ac..66adbd4b 100644 --- a/src/bitarray.jl +++ b/src/bitarray.jl @@ -12,7 +12,7 @@ # - _unsafe_acquire_impl! for Bit - Raw BitArray acquisition with caching # - DisabledPool fallbacks for Bit type # - empty!(::BitTypedPool) - State management (clearing pool storage) -# - _check_bitchunks_overlap - Safety validation for POOL_DEBUG mode +# - _check_bitchunks_overlap - Safety validation for S=1 runtime check mode # - Display helpers: _default_type_name, _vector_bytes, _count_label, _show_type_name # # Design Decision: Unified BitArray Return Type @@ -195,7 +195,7 @@ function Base.empty!(tp::BitTypedPool) end # ============================================================================== -# Safety Validation (POOL_DEBUG mode) +# Safety Validation (S=1 runtime check mode) # ============================================================================== # Check if BitArray chunks overlap with the pool's BitTypedPool storage diff --git a/src/debug.jl b/src/debug.jl index 83eed243..4469c164 100644 --- a/src/debug.jl +++ b/src/debug.jl @@ -1,21 +1,7 @@ # ============================================================================== -# Debugging & Safety (POOL_DEBUG escape detection) +# Debugging & Safety (Runtime escape detection, RUNTIME_CHECK >= 1) # ============================================================================== -""" - POOL_DEBUG - -Legacy flag for escape detection. Superseded by [`POOL_SAFETY_LV`](@ref). - -Setting `POOL_DEBUG[] = true` enables escape detection at `@with_pool` scope exit -(equivalent to `POOL_SAFETY_LV[] >= 2` behavior). Both flags are checked independently. - -For new code, prefer `POOL_SAFETY_LV[] = 2`. - -Default: `false` -""" -const POOL_DEBUG = Ref(false) - function _validate_pool_return(val, pool::AdaptiveArrayPool) # 0. Check BitArray / BitVector (bit-packed storage) if val isa BitArray @@ -117,23 +103,22 @@ end PoolRuntimeEscapeError <: Exception Thrown at runtime when `_validate_pool_return` detects a pool-backed array -escaping from an `@with_pool` scope (requires `POOL_SAFETY_LV[] >= 2`). +escaping from an `@with_pool` scope (requires `RUNTIME_CHECK >= 1`). This is the runtime counterpart of [`PoolEscapeError`](@ref) (compile-time). """ struct PoolRuntimeEscapeError <: Exception val_summary::String pool_eltype::String - callsite::Union{Nothing, String} # acquire location (LV ≥ 3) - return_site::Union{Nothing, String} # return location (LV ≥ 3) + callsite::Union{Nothing, String} # acquire location (S ≥ 1) + return_site::Union{Nothing, String} # return location (S ≥ 1) end function Base.showerror(io::IO, e::PoolRuntimeEscapeError) has_callsite = e.callsite !== nothing - lv_label = has_callsite ? "POOL_SAFETY_LV ≥ 3" : "POOL_SAFETY_LV ≥ 2" printstyled(io, "PoolEscapeError"; color = :red, bold = true) - printstyled(io, " (runtime, ", lv_label, ")"; color = :light_black) + printstyled(io, " (runtime, RUNTIME_CHECK >= 1)"; color = :light_black) println(io) println(io) @@ -187,13 +172,7 @@ function Base.showerror(io::IO, e::PoolRuntimeEscapeError) printstyled(io, "collect()"; bold = true) printstyled(io, " to return an owned copy, or compute a scalar result.\n"; color = :light_black) - return if !has_callsite - println(io) - printstyled(io, " Tip: "; bold = true) - printstyled(io, "set "; color = :light_black) - printstyled(io, "POOL_SAFETY_LV[] = 3"; bold = true) - printstyled(io, " for acquire!() call-site tracking.\n"; color = :light_black) - end + return nothing end Base.showerror(io::IO, e::PoolRuntimeEscapeError, ::Any; backtrace = true) = showerror(io, e) @@ -245,7 +224,7 @@ _validate_pool_return(val, ::DisabledPool) = nothing _validate_pool_return(val, ::AbstractArrayPool) = nothing # ============================================================================== -# Poisoning: Fill released vectors with sentinel values (POOL_SAFETY_LV >= 2) +# Poisoning: Fill released vectors with sentinel values (S >= 1) # ============================================================================== # # Poisons backing vectors with detectable values (NaN, typemax) before @@ -266,7 +245,7 @@ _poison_fill!(v::BitVector) = fill!(v, true) _poison_released_vectors!(tp::AbstractTypedPool, old_n_active) Fill released backing vectors (indices `n_active+1:old_n_active`) with sentinel -values. Called from `_invalidate_released_slots!` when `POOL_SAFETY_LV[] >= 2`, +values. Called from `_invalidate_released_slots!` when `S >= 1`, before `resize!` zeroes the lengths. """ @noinline function _poison_released_vectors!(tp::AbstractTypedPool, old_n_active::Int) @@ -307,7 +286,7 @@ function _shorten_location(location::String) end # ============================================================================== -# Borrow Registry: Call-site tracking for acquire! (POOL_SAFETY_LV >= 3) +# Borrow Registry: Call-site tracking for acquire! (S >= 1) # ============================================================================== # # Records where each acquire! call originated (file:line) so escape errors @@ -319,7 +298,7 @@ end _record_borrow_from_pending!(pool, tp) Record the pending callsite for the most recently claimed slot in `tp`. -Called from `_acquire_impl!` / `_unsafe_acquire_impl!` when `POOL_SAFETY_LV[] >= 3`. +Called from `_acquire_impl!` / `_unsafe_acquire_impl!` when `S >= 1`. """ @noinline function _record_borrow_from_pending!(pool::AdaptiveArrayPool, tp::AbstractTypedPool) callsite = pool._pending_callsite @@ -338,7 +317,7 @@ end _lookup_borrow_callsite(pool, v) -> Union{Nothing, String} Look up the callsite string for a pool backing vector. Returns `nothing` if -no borrow was recorded (LV < 3 or non-macro path without callsite info). +no borrow was recorded (S=0 or non-macro path without callsite info). """ @noinline function _lookup_borrow_callsite(pool::AdaptiveArrayPool, v)::Union{Nothing, String} log = pool._borrow_log diff --git a/src/legacy/bitarray.jl b/src/legacy/bitarray.jl index dcf3efb7..95b26d35 100644 --- a/src/legacy/bitarray.jl +++ b/src/legacy/bitarray.jl @@ -12,7 +12,7 @@ # - _unsafe_acquire_impl! for Bit - Raw BitArray acquisition with caching # - DisabledPool fallbacks for Bit type # - empty!(::BitTypedPool) - State management (clearing pool storage) -# - _check_bitchunks_overlap - Safety validation for POOL_DEBUG mode +# - _check_bitchunks_overlap - Safety validation for S=1 runtime check mode # - Display helpers: _default_type_name, _vector_bytes, _count_label, _show_type_name # # Design Decision: Unified BitArray Return Type @@ -234,7 +234,7 @@ function Base.empty!(tp::BitTypedPool) end # ============================================================================== -# Safety Validation (POOL_DEBUG mode) +# Safety Validation (S=1 runtime check mode) # ============================================================================== # Check if BitArray chunks overlap with the pool's BitTypedPool storage diff --git a/src/legacy/state.jl b/src/legacy/state.jl index 7d6d1868..75c9f744 100644 --- a/src/legacy/state.jl +++ b/src/legacy/state.jl @@ -248,7 +248,7 @@ Decrements _current_depth once after all types are rewound. end # ============================================================================== -# Safety: Structural Invalidation on Rewind (POOL_SAFETY_LV >= 1) +# Safety: Structural Invalidation on Rewind (S >= 1) # ============================================================================== # # When released, backing vectors are resize!'d to 0 and cached Array/BitArray @@ -261,15 +261,15 @@ end _invalidate_released_slots!(::AbstractTypedPool, ::Int, ::Int) = nothing _invalidate_released_slots!(::AbstractTypedPool, ::Int) = nothing # legacy 2-arg compat -@noinline function _invalidate_released_slots!(tp::TypedPool{T}, old_n_active::Int, S::Int = POOL_SAFETY_LV[]) where {T} +@noinline function _invalidate_released_slots!(tp::TypedPool{T}, old_n_active::Int, S::Int) where {T} new_n = tp.n_active - # Level 2+: poison vectors with NaN/sentinel before structural invalidation. + # S=1: poison vectors with NaN/sentinel before structural invalidation. # Especially useful on legacy (1.10) where unsafe_acquire! Array wrappers # can't be structurally invalidated (Array is a C struct, no setfield!). - if S >= 2 + if S >= 1 _poison_released_vectors!(tp, old_n_active) end - # Level 1+: resize backing vectors to length 0 (invalidates SubArrays from acquire!) + # S=1: resize backing vectors to length 0 (invalidates SubArrays from acquire!) # Note: Array wrapper invalidation (setfield! :size) requires Julia 1.11+. # On legacy (1.10), only SubArray invalidation via resize! is available. for i in (new_n + 1):old_n_active @@ -283,13 +283,13 @@ _invalidate_released_slots!(::AbstractTypedPool, ::Int) = nothing # legacy 2-ar return nothing end -@noinline function _invalidate_released_slots!(tp::BitTypedPool, old_n_active::Int, S::Int = POOL_SAFETY_LV[]) +@noinline function _invalidate_released_slots!(tp::BitTypedPool, old_n_active::Int, S::Int) new_n = tp.n_active - # Level 2+: poison BitVectors (all bits set to true) - if S >= 2 + # S=1: poison BitVectors (all bits set to true) + if S >= 1 _poison_released_vectors!(tp, old_n_active) end - # Level 1+: resize backing BitVectors to length 0 + # S=1: resize backing BitVectors to length 0 for i in (new_n + 1):old_n_active @inbounds resize!(tp.vectors[i], 0) end @@ -324,10 +324,9 @@ end # Internal helper for rewind with orphan cleanup (works for any AbstractTypedPool) # Uses 1-based sentinel pattern: no isempty checks needed (sentinel [0] guarantees non-empty) # -# S parameter: safety level. When called from AdaptiveArrayPool{S} callers, S is a -# compile-time constant → `S >= 1` dead-code-eliminates the invalidation branch at S=0. -# Default S = POOL_SAFETY_LV[] preserves backward compat for CUDA ext and legacy callers. -@inline function _rewind_typed_pool!(tp::AbstractTypedPool, current_depth::Int, S::Int = POOL_SAFETY_LV[]) +# S parameter: runtime check level (0=off, 1=on). When called from AdaptiveArrayPool{S} +# callers, S is a compile-time constant → `S >= 1` dead-code-eliminates at S=0. +@inline function _rewind_typed_pool!(tp::AbstractTypedPool, current_depth::Int, S::Int) # 1. Orphaned Checkpoints Cleanup # If there are checkpoints from deeper scopes (depth > current), pop them first. @@ -557,7 +556,7 @@ end Reset state without clearing allocated storage. Sets `n_active = 0` and restores checkpoint stacks to sentinel state. """ -function reset!(tp::AbstractTypedPool, S::Int = POOL_SAFETY_LV[]) +function reset!(tp::AbstractTypedPool, S::Int) _old_n_active = tp.n_active tp.n_active = 0 # Restore sentinel values (1-based sentinel pattern) diff --git a/src/legacy/types.jl b/src/legacy/types.jl index 212579c7..d365bcd3 100644 --- a/src/legacy/types.jl +++ b/src/legacy/types.jl @@ -66,67 +66,35 @@ end # ============================================================================== # # Safety is controlled per-pool via the type parameter S in AdaptiveArrayPool{S}. -# S encodes the safety level (0-3), enabling dead-code elimination at compile time. +# S is binary: 0 (off) or 1 (on), enabling dead-code elimination at compile time. # -# 0 = off — zero overhead (default) -# 1 = guard — structural invalidation on rewind (resize + setfield!) -# 2 = full — guard + escape detection + poisoning -# 3 = debug — full + borrow registry (call-site tracking) +# 0 = off — zero overhead (default) +# 1 = on — full runtime checks (invalidation, poisoning, escape detection, borrow tracking) # -# DEFAULT_SAFETY_LV sets the initial S for new pools. -# POOL_SAFETY_LV is the runtime Ref read by AdaptiveArrayPool() constructor. -# Use set_safety_level!() to replace the task-local pool at runtime. +# Int type allows future compile-time check levels (like -O0/-O1/-O2). +# Currently only 0 and 1 are defined. LocalPreferences.toml accepts both +# Bool (true/false) and Int (0/1). +# +using Preferences: @load_preference + +_normalize_runtime_check(v::Bool) = Int(v) +_normalize_runtime_check(v::Integer) = clamp(Int(v), 0, 1) """ - DEFAULT_SAFETY_LV::Int + RUNTIME_CHECK::Int + +Compile-time constant controlling the runtime safety check level. -Compile-time default safety level for new pool instances. +- `0` — off (zero overhead, default) +- `1` — full checks (invalidation, poisoning, escape detection, borrow tracking) Set via `LocalPreferences.toml`: ```toml [AdaptiveArrayPools] -safety_level = 1 +runtime_check = true # or 1 ``` - -Levels: `0` (off, default), `1` (guard), `2` (full), `3` (debug). - -Legacy fallback: reads `pool_checks` preference if `safety_level` is not set -(`true` → 1, `false` → 0). -""" -const DEFAULT_SAFETY_LV = let - sl = @load_preference("safety_level", nothing) - if sl !== nothing - sl::Int - else - # Legacy: pool_checks=true → safety_level=1 - pc = @load_preference("pool_checks", nothing) - pc !== nothing ? (pc::Bool ? 1 : 0) : 0 - end -end - -""" - STATIC_POOL_CHECKS::Bool - -Compile-time constant derived from `DEFAULT_SAFETY_LV > 0`. -Retained for backward compatibility. New code should use `_safety_level(pool) >= N`. -""" -const STATIC_POOL_CHECKS = DEFAULT_SAFETY_LV > 0 - -""" - POOL_SAFETY_LV - -Runtime safety level. Read by `AdaptiveArrayPool()` constructor to determine -the `S` parameter of newly created pools. - -- `0`: Off — zero overhead -- `1`: Guard — structural invalidation on rewind (resize + setfield!) -- `2`: Full — guard + escape detection on scope exit + poisoning -- `3`: Debug — full + borrow registry (acquire call-site tracking) - -Initial value: `DEFAULT_SAFETY_LV` (default `0`). -Use `set_safety_level!()` to change the task-local pool at runtime. """ -const POOL_SAFETY_LV = Ref{Int}(DEFAULT_SAFETY_LV) +const RUNTIME_CHECK = _normalize_runtime_check(@load_preference("runtime_check", 0)) # ============================================================================== # Abstract Type Hierarchy (for extensibility) @@ -425,11 +393,11 @@ const _TYPE_BITS_MASK = UInt16(0x00FF) # bits 0-7: fixed-slot type bits Multi-type memory pool with fixed slots for common types and IdDict fallback for others. Zero allocation after warmup. NOT thread-safe - use one pool per Task. -The type parameter `S::Int` encodes the safety level (0-3). Inside `@inline` call chains, -`S` is a compile-time constant — safety checks like `S >= 1` are eliminated by dead-code -elimination when `S = 0`, achieving true zero overhead. +The type parameter `S::Int` encodes the runtime check mode (0 = off, 1 = on). +Inside `@inline` call chains, `S` is a compile-time constant — safety checks are +eliminated by dead-code elimination when `S = 0`, achieving true zero overhead. -See also: [`_safety_level`], [`_make_pool`], [`set_safety_level!`] +See also: [`_runtime_check`], [`_make_pool`], [`RUNTIME_CHECK`] """ mutable struct AdaptiveArrayPool{S} <: AbstractArrayPool # Fixed Slots: common types with zero lookup overhead @@ -450,7 +418,7 @@ mutable struct AdaptiveArrayPool{S} <: AbstractArrayPool _touched_type_masks::Vector{UInt16} # Per-depth: which fixed slots were touched + mode flags _touched_has_others::Vector{Bool} # Per-depth: any non-fixed-slot type touched? - # Borrow registry (POOL_SAFETY_LV >= 3 only) + # Borrow registry (S = 1 only) _pending_callsite::String # "" = no pending; set by macro before acquire _pending_return_site::String # "" = no pending; set by macro before validate _borrow_log::Union{Nothing, IdDict{Any, String}} # vector_obj => callsite string @@ -472,45 +440,46 @@ function AdaptiveArrayPool{S}() where {S} [false], # _touched_has_others: sentinel (no others) "", # _pending_callsite: no pending "", # _pending_return_site: no pending - nothing # _borrow_log: lazily created at LV >= 3 + nothing # _borrow_log: lazily created at S=1 ) end -"""Create pool at the current `POOL_SAFETY_LV[]` level.""" -AdaptiveArrayPool() = _make_pool(POOL_SAFETY_LV[]) +"""Create pool with the default `RUNTIME_CHECK` level.""" +AdaptiveArrayPool() = AdaptiveArrayPool{RUNTIME_CHECK}() """ - _safety_level(pool) -> Int + _runtime_check(pool) -> Bool -Return the safety level of a pool. Compile-time constant for `AdaptiveArrayPool{S}`. +Return whether runtime safety checks are enabled for `pool` (i.e., `S >= 1`). +Compile-time constant for `AdaptiveArrayPool{S}` — dead-code eliminated when `S = 0`. """ -@inline _safety_level(::AdaptiveArrayPool{S}) where {S} = S -@inline _safety_level(::AbstractArrayPool) = POOL_SAFETY_LV[] # fallback +@inline _runtime_check(::AdaptiveArrayPool{0}) = false +@inline _runtime_check(::AdaptiveArrayPool) = true # S >= 1 """ - _make_pool(s::Int) -> AdaptiveArrayPool{s} + _make_pool(level) -> AdaptiveArrayPool -Function barrier: converts runtime `Int` to concrete `AdaptiveArrayPool{S}`. -Levels outside 0-3 are clamped (≤0 → 0, ≥3 → 3). +Function barrier: converts runtime check level to concrete `AdaptiveArrayPool{S}`. +Accepts `Bool` (`true`→1, `false`→0) or `Int` (used directly as S). """ -@noinline function _make_pool(s::Int) - s <= 0 && return AdaptiveArrayPool{0}() - s == 1 && return AdaptiveArrayPool{1}() - s == 2 && return AdaptiveArrayPool{2}() - return AdaptiveArrayPool{3}() +_make_pool(runtime_check::Bool) = _make_pool(Int(runtime_check)) +@noinline function _make_pool(S::Int) + S == 0 && return AdaptiveArrayPool{0}() + return AdaptiveArrayPool{1}() end """ - _make_pool(s::Int, old::AdaptiveArrayPool) -> AdaptiveArrayPool{s} + _make_pool(level, old::AdaptiveArrayPool) -> AdaptiveArrayPool -Create a new pool at safety level `s`, transferring cached arrays and scope state -from `old`. Only reference copies — no memory allocation for the underlying buffers. +Create a new pool, transferring cached arrays and scope state from `old`. +Only reference copies — no memory allocation for the underlying buffers. Transferred: all TypedPool/BitTypedPool slots, `others`, depth & touch tracking. Reset: `_pending_callsite/return_site` (transient macro state), - `_borrow_log` (created fresh when `s >= 3`). + `_borrow_log` (created fresh when S >= 1). """ -@noinline function _make_pool(s::Int, old::AdaptiveArrayPool) +_make_pool(runtime_check::Bool, old::AdaptiveArrayPool) = _make_pool(Int(runtime_check), old) +@noinline function _make_pool(level::Int, old::AdaptiveArrayPool) _new(::Val{S}) where {S} = AdaptiveArrayPool{S}( old.float64, old.float32, old.int64, old.int32, old.complexf64, old.complexf32, old.bool, old.bits, @@ -520,12 +489,10 @@ Reset: `_pending_callsite/return_site` (transient macro state), old._touched_has_others, "", # _pending_callsite: reset "", # _pending_return_site: reset - S >= 3 ? IdDict{Any, String}() : nothing # _borrow_log + S >= 1 ? IdDict{Any, String}() : nothing # _borrow_log ) - s <= 0 && return _new(Val(0)) - s == 1 && return _new(Val(1)) - s == 2 && return _new(Val(2)) - return _new(Val(3)) + level == 0 && return _new(Val(0)) + return _new(Val(1)) end # ============================================================================== @@ -590,12 +557,12 @@ end """ _set_pending_callsite!(pool, msg::String) -Record a pending callsite string for borrow tracking (safety level ≥ 3). +Record a pending callsite string for borrow tracking (S=1). Only sets the callsite if no prior callsite is pending (macro-injected ones take priority). -Compiles to no-op when `S < 3`. +Compiles to no-op when `S=0`. """ @inline function _set_pending_callsite!(pool::AdaptiveArrayPool{S}, msg::String) where {S} - S >= 3 && isempty(pool._pending_callsite) && (pool._pending_callsite = msg) + S >= 1 && isempty(pool._pending_callsite) && (pool._pending_callsite = msg) return nothing end @inline _set_pending_callsite!(::AbstractArrayPool, ::String) = nothing @@ -603,12 +570,12 @@ end """ _maybe_record_borrow!(pool, tp::AbstractTypedPool) -Flush the pending callsite into the borrow log (safety level ≥ 3). +Flush the pending callsite into the borrow log (S=1). Delegates to `_record_borrow_from_pending!` (defined in `debug.jl`). -Compiles to no-op when `S < 3`. +Compiles to no-op when `S=0`. """ @inline function _maybe_record_borrow!(pool::AdaptiveArrayPool{S}, tp::AbstractTypedPool) where {S} - S >= 3 && _record_borrow_from_pending!(pool, tp) + S >= 1 && _record_borrow_from_pending!(pool, tp) return nothing end @inline _maybe_record_borrow!(::AbstractArrayPool, ::AbstractTypedPool) = nothing diff --git a/src/macros.jl b/src/macros.jl index f33707fc..9d3efed6 100644 --- a/src/macros.jl +++ b/src/macros.jl @@ -518,14 +518,14 @@ function _fix_generated_lnn!(expr, source::Union{LineNumberNode, Nothing}) end # ============================================================================== -# Internal: Union Splitting Wrapper Helper +# Internal: Compile-Time Type Assertion Helper # ============================================================================== """ _pool_type_for_backend(::Val{B}) -> Type Returns the concrete pool type for a given backend, used at macro expansion time -to generate closureless union splitting. Extensions override this for their backends. +to generate direct type assertions. Extensions override this for their backends. CPU returns `AdaptiveArrayPool`, CUDA extension returns `CuAdaptiveArrayPool`. """ @@ -535,11 +535,11 @@ _pool_type_for_backend(::Val{B}) where {B} = nothing # unregistered backend — """ _wrap_with_dispatch(pool_name_esc, pool_getter, inner_body; backend=:cpu) -Closureless union splitting: generates `let _raw = getter; if _raw isa PoolType{0} ...` -chain that narrows `pool_name` to concrete `PoolType{S}` without a closure. +Direct type assertion: generates `let pool = getter::PoolType{RUNTIME_CHECK}`. -Eliminates Core.Box boxing that occurs when closure-based `_dispatch_pool_scope` -gets inlined into outer callers crossing try/finally boundaries. +Since `RUNTIME_CHECK` is a compile-time `const Int`, the pool type parameter S +is resolved at compile time. `_runtime_check(pool)` returns a compile-time Bool, +enabling dead-code elimination of all safety branches when `RUNTIME_CHECK = 0`. The pool type is resolved at macro expansion time via `_pool_type_for_backend`, which extensions override (e.g., CUDA adds `CuAdaptiveArrayPool`). @@ -547,24 +547,14 @@ which extensions override (e.g., CUDA adds `CuAdaptiveArrayPool`). function _wrap_with_dispatch(pool_name_esc, pool_getter, inner_body; backend::Symbol = :cpu) PoolType = _pool_type_for_backend(Val{backend}()) if PoolType === nothing - # Unregistered backend: fall back to closure-based dispatch. - # Runtime will error in _get_pool_for_backend if extension isn't loaded. - return :( - $(_DISPATCH_POOL_SCOPE_REF)($pool_getter) do $pool_name_esc - $inner_body - end - ) + # Unregistered backend: no type assertion, runtime will error in pool getter. + return Expr(:let, Expr(:(=), pool_name_esc, pool_getter), inner_body) end _PT = GlobalRef(parentmodule(PoolType), nameof(PoolType)) - raw = gensym(:_raw_pool) - # Fallback: S=3 (last branch, no condition needed) - chain = Expr(:let, Expr(:(=), pool_name_esc, :($raw::$_PT{3})), inner_body) - for s in 2:-1:0 - concrete_t = :($_PT{$s}) - branch_body = Expr(:let, Expr(:(=), pool_name_esc, :($raw::$concrete_t)), inner_body) - chain = Expr(:if, :($raw isa $concrete_t), branch_body, chain) - end - return Expr(:let, Expr(:(=), raw, pool_getter), chain) + _RC = GlobalRef(@__MODULE__, :RUNTIME_CHECK) + # RUNTIME_CHECK is const Int → compiler resolves to literal S, zero branching. + concrete_t = :($_PT{$_RC}) + return Expr(:let, Expr(:(=), pool_name_esc, :($pool_getter::$concrete_t)), inner_body) end # ============================================================================== @@ -610,7 +600,7 @@ function _generate_pool_code(pool_name, expr, force_enable; source::Union{LineNu transformed_expr = use_typed ? _transform_acquire_calls(expr, pool_name) : expr # Inject borrow callsite recording + return validation. - # Always injected — _safety_level(pool) gates at runtime (dead-code-eliminated at S=0). + # Always injected — _runtime_check(pool) gates at runtime (dead-code-eliminated when false). transformed_expr = _inject_pending_callsite(transformed_expr, pool_name, expr) transformed_expr = _transform_return_stmts(transformed_expr, pool_name) @@ -626,12 +616,12 @@ function _generate_pool_code(pool_name, expr, force_enable; source::Union{LineNu rewind_call = _generate_lazy_rewind_call(esc(pool_name)) end - # Build the inner body (runs inside _dispatch_pool_scope closure where pool is concrete) + # Build the inner body (runs inside let-block where pool has concrete type) inner = quote $checkpoint_call try local _result = $(esc(transformed_expr)) - if ($_SAFETY_LEVEL_REF($(esc(pool_name))) >= 2 || $_POOL_DEBUG_REF[]) + if $_RUNTIME_CHECK_REF($(esc(pool_name))) $_validate_pool_return(_result, $(esc(pool_name))) end _result @@ -721,7 +711,7 @@ function _generate_pool_code_with_backend(backend::Symbol, pool_name, expr, forc $checkpoint_call try local _result = $(esc(transformed_expr)) - if ($_SAFETY_LEVEL_REF($(esc(pool_name))) >= 2 || $_POOL_DEBUG_REF[]) + if $_RUNTIME_CHECK_REF($(esc(pool_name))) $_validate_pool_return(_result, $(esc(pool_name))) end _result @@ -783,7 +773,7 @@ function _generate_pool_code_with_backend(backend::Symbol, pool_name, expr, forc $checkpoint_call try local _result = $(esc(transformed_expr)) - if ($_SAFETY_LEVEL_REF($(esc(pool_name))) >= 2 || $_POOL_DEBUG_REF[]) + if $_RUNTIME_CHECK_REF($(esc(pool_name))) $_validate_pool_return(_result, $(esc(pool_name))) end _result @@ -853,7 +843,7 @@ function _generate_function_pool_code_with_backend(backend::Symbol, pool_name, f local _result = begin $(esc(transformed_body)) end - if ($_SAFETY_LEVEL_REF($(esc(pool_name))) >= 2 || $_POOL_DEBUG_REF[]) + if $_RUNTIME_CHECK_REF($(esc(pool_name))) $_validate_pool_return(_result, $(esc(pool_name))) end _result @@ -938,7 +928,7 @@ function _generate_function_pool_code(pool_name, func_def, force_enable, disable local _result = begin $(esc(transformed_body)) end - if ($_SAFETY_LEVEL_REF($(esc(pool_name))) >= 2 || $_POOL_DEBUG_REF[]) + if $_RUNTIME_CHECK_REF($(esc(pool_name))) $_validate_pool_return(_result, $(esc(pool_name))) end _result @@ -955,7 +945,7 @@ function _generate_function_pool_code(pool_name, func_def, force_enable, disable local _result = begin $(esc(transformed_body)) end - if ($_SAFETY_LEVEL_REF($(esc(pool_name))) >= 2 || $_POOL_DEBUG_REF[]) + if $_RUNTIME_CHECK_REF($(esc(pool_name))) $_validate_pool_return(_result, $(esc(pool_name))) end _result @@ -1435,7 +1425,7 @@ function _transform_acquire_calls(expr, pool_name) end # ============================================================================== -# Internal: Borrow Callsite Injection (POOL_SAFETY_LV >= 3) +# Internal: Borrow Callsite Injection (S = 1) # ============================================================================== # # Second-pass AST transformation that inserts `pool._pending_callsite = "file:line"` @@ -1444,11 +1434,9 @@ end # # Works with both typed path (_*_impl! GlobalRefs) and dynamic path (original # acquire!/zeros!/etc. calls). Always injected — gated at runtime by -# `_safety_level(pool) >= 3 || POOL_DEBUG[]` (dead-code-eliminated at S<3). +# `_runtime_check(pool)` (dead-code-eliminated when S=0). -const _POOL_SAFETY_LV_REF = GlobalRef(@__MODULE__, :POOL_SAFETY_LV) -const _DISPATCH_POOL_SCOPE_REF = GlobalRef(@__MODULE__, :_dispatch_pool_scope) -const _SAFETY_LEVEL_REF = GlobalRef(@__MODULE__, :_safety_level) +const _RUNTIME_CHECK_REF = GlobalRef(@__MODULE__, :_runtime_check) """Set of all transformed `_*_impl!` function names (GlobalRef targets).""" const _IMPL_FUNC_NAMES = Set{Symbol}( @@ -1506,7 +1494,7 @@ end _inject_pending_callsite(expr, pool_name, original_expr=expr) -> Expr Walk block-level statements, track `LineNumberNode`s, and insert -`POOL_SAFETY_LV[] >= 3 && (pool._pending_callsite = "file:line\\nexpr")` +`_runtime_check(pool) && (pool._pending_callsite = "file:line\\nexpr")` before each statement containing a pool acquire call. When `original_expr` differs from `expr` (i.e., after `_transform_acquire_calls`), @@ -1537,11 +1525,7 @@ function _inject_pending_callsite(expr, pool_name, original_expr = expr) "$(current_lnn.file):$(current_lnn.line)\n$(expr_text)" inject = Expr( :&&, - Expr( - :||, - Expr(:call, :>=, Expr(:call, _SAFETY_LEVEL_REF, pool_name), 3), - Expr(:ref, _POOL_DEBUG_REF) - ), + Expr(:call, _RUNTIME_CHECK_REF, pool_name), Expr( :(=), Expr(:., pool_name, QuoteNode(:_pending_callsite)), @@ -1566,7 +1550,7 @@ function _inject_pending_callsite(expr, pool_name, original_expr = expr) end # ============================================================================== -# Internal: Return Statement Validation (POOL_SAFETY_LV >= 2) +# Internal: Return Statement Validation (S = 1) # ============================================================================== # # Transforms `return expr` → `begin local _ret = expr; validate(_ret); return _ret end` @@ -1577,14 +1561,13 @@ end # Stops recursion at :function and :-> boundaries (nested function return statements # belong to the inner function, not the @with_pool scope). -const _POOL_DEBUG_REF = GlobalRef(@__MODULE__, :POOL_DEBUG) const _VALIDATE_POOL_RETURN_REF = GlobalRef(@__MODULE__, :_validate_pool_return) """ _transform_return_stmts(expr, pool_name) -> Expr Walk AST and wrap explicit `return value` statements with escape validation. -Generates: `local _ret = value; if (LV≥2 || DEBUG) validate(_ret, pool); end; return _ret` +Generates: `local _ret = value; if _runtime_check(pool) validate(_ret, pool); end; return _ret` Does NOT recurse into nested `:function` or `:->` expressions (inner functions have their own `return` semantics). @@ -1607,7 +1590,7 @@ function _transform_return_stmts(expr, pool_name, current_lnn = nothing) value_expr = _transform_return_stmts(value_expr, pool_name, current_lnn) retvar = gensym(:_pool_ret) - # Build return-site string for LV ≥ 3 display (e.g. "file:line\nreturn v") + # Build return-site string for S=1 display (e.g. "file:line\nreturn v") return_site_str = if current_lnn !== nothing "$(current_lnn.file):$(current_lnn.line)\n$(string(expr))" else @@ -1620,11 +1603,7 @@ function _transform_return_stmts(expr, pool_name, current_lnn = nothing) :block, Expr( :&&, - Expr( - :||, - Expr(:call, :>=, Expr(:call, _SAFETY_LEVEL_REF, pool_name), 3), - Expr(:ref, _POOL_DEBUG_REF) - ), + Expr(:call, _RUNTIME_CHECK_REF, pool_name), Expr( :(=), Expr(:., pool_name, QuoteNode(:_pending_return_site)), @@ -1642,11 +1621,7 @@ function _transform_return_stmts(expr, pool_name, current_lnn = nothing) Expr(:local, Expr(:(=), retvar, value_expr)), Expr( :if, - Expr( - :||, - Expr(:call, :>=, Expr(:call, _SAFETY_LEVEL_REF, pool_name), 2), - Expr(:ref, _POOL_DEBUG_REF) - ), + Expr(:call, _RUNTIME_CHECK_REF, pool_name), validate_expr ), Expr(:return, retvar) diff --git a/src/state.jl b/src/state.jl index 9903c0b9..92fd515c 100644 --- a/src/state.jl +++ b/src/state.jl @@ -231,7 +231,7 @@ Decrements _current_depth once after all types are rewound. end # ============================================================================== -# Safety: Structural Invalidation on Rewind (POOL_SAFETY_LV >= 1) +# Safety: Structural Invalidation on Rewind (S >= 1) # ============================================================================== # # When released, backing vectors are resize!'d to 0 and cached Array/BitArray @@ -255,13 +255,13 @@ _invalidate_released_slots!(::AbstractTypedPool, ::Int) = nothing # legacy 2-ar return ntuple(_ -> 0, N) end -@noinline function _invalidate_released_slots!(tp::TypedPool{T}, old_n_active::Int, S::Int = POOL_SAFETY_LV[]) where {T} +@noinline function _invalidate_released_slots!(tp::TypedPool{T}, old_n_active::Int, S::Int) where {T} new_n = tp.n_active - # Level 2+: poison vectors with NaN/sentinel before structural invalidation - if S >= 2 + # S=1: poison vectors with NaN/sentinel before structural invalidation + if S >= 1 _poison_released_vectors!(tp, old_n_active) end - # Level 1+: resize backing vectors to length 0 (invalidates SubArrays from acquire!) + # S=1: resize backing vectors to length 0 (invalidates SubArrays from acquire!) for i in (new_n + 1):old_n_active @inbounds resize!(tp.vectors[i], 0) end @@ -279,13 +279,13 @@ end return nothing end -@noinline function _invalidate_released_slots!(tp::BitTypedPool, old_n_active::Int, S::Int = POOL_SAFETY_LV[]) +@noinline function _invalidate_released_slots!(tp::BitTypedPool, old_n_active::Int, S::Int) new_n = tp.n_active - # Level 2+: poison BitVectors (all bits set to true) - if S >= 2 + # S=1: poison BitVectors (all bits set to true) + if S >= 1 _poison_released_vectors!(tp, old_n_active) end - # Level 1+: resize backing BitVectors to length 0 (invalidates chunks) + # S=1: resize backing BitVectors to length 0 (invalidates chunks) for i in (new_n + 1):old_n_active @inbounds resize!(tp.vectors[i], 0) end @@ -312,10 +312,9 @@ end # Internal helper for rewind with orphan cleanup (works for any AbstractTypedPool) # Uses 1-based sentinel pattern: no isempty checks needed (sentinel [0] guarantees non-empty) # -# S parameter: safety level. When called from AdaptiveArrayPool{S} callers, S is a -# compile-time constant → `S >= 1` dead-code-eliminates the invalidation branch at S=0. -# Default S = POOL_SAFETY_LV[] preserves backward compat for CUDA ext and legacy callers. -@inline function _rewind_typed_pool!(tp::AbstractTypedPool, current_depth::Int, S::Int = POOL_SAFETY_LV[]) +# S parameter: runtime check level (0=off, 1=on). When called from AdaptiveArrayPool{S} +# callers, S is a compile-time constant → `S >= 1` dead-code-eliminates at S=0. +@inline function _rewind_typed_pool!(tp::AbstractTypedPool, current_depth::Int, S::Int) # 1. Orphaned Checkpoints Cleanup # If there are checkpoints from deeper scopes (depth > current), pop them first. @@ -542,7 +541,7 @@ end Reset state without clearing allocated storage. Sets `n_active = 0` and restores checkpoint stacks to sentinel state. """ -function reset!(tp::AbstractTypedPool, S::Int = POOL_SAFETY_LV[]) +function reset!(tp::AbstractTypedPool, S::Int) _old_n_active = tp.n_active tp.n_active = 0 # Restore sentinel values (1-based sentinel pattern) diff --git a/src/task_local_pool.jl b/src/task_local_pool.jl index d316a703..6dc406e7 100644 --- a/src/task_local_pool.jl +++ b/src/task_local_pool.jl @@ -82,8 +82,9 @@ Retrieves (or creates) the `AdaptiveArrayPool` for the current Task. Each Task gets its own pool instance via `task_local_storage()`, ensuring thread safety without locks. -Returns the pool as-is (type `Any` from task_local_storage). -Use `_dispatch_pool_scope` in macro-generated code to narrow to concrete `AdaptiveArrayPool{S}`. +The pool type is `AdaptiveArrayPool{S}` where `S` is determined by +the compile-time constant `RUNTIME_CHECK::Int`. Macro-generated code +type-asserts directly to `AdaptiveArrayPool{RUNTIME_CHECK}`. """ @inline function get_task_local_pool() # 1. Fast Path: Try to get existing pool @@ -100,88 +101,6 @@ Use `_dispatch_pool_scope` in macro-generated code to narrow to concrete `Adapti return pool::AdaptiveArrayPool end -# ============================================================================== -# Union Splitting Dispatcher + Safety Level Control -# ============================================================================== -# -# AdaptiveArrayPool{S} is parametric on both modern (≥1.11) and legacy (≤1.10). -# Union splitting narrows to concrete type for dead-code elimination of safety branches. - -""" - _dispatch_pool_scope(f, pool_any) - -Union splitting barrier: converts abstract pool → concrete `AdaptiveArrayPool{S}`. - -Inside `f`, the pool argument has concrete type, enabling: -- `_safety_level(pool)` → compile-time constant S -- Dead-code elimination of safety branches at S=0 -- Zero-allocation try/finally (no Core.Box) - -Called from macro-generated code as: -```julia -_dispatch_pool_scope(get_task_local_pool()) do pool - checkpoint!(pool) - try ... finally rewind!(pool) end -end -``` -""" -@inline function _dispatch_pool_scope(f, pool_any) - if pool_any isa AdaptiveArrayPool{0} - return f(pool_any::AdaptiveArrayPool{0}) - elseif pool_any isa AdaptiveArrayPool{1} - return f(pool_any::AdaptiveArrayPool{1}) - elseif pool_any isa AdaptiveArrayPool{2} - return f(pool_any::AdaptiveArrayPool{2}) - elseif pool_any isa AdaptiveArrayPool{3} - return f(pool_any::AdaptiveArrayPool{3}) - else - # Non-CPU pools (e.g. CuAdaptiveArrayPool): pass through as-is. - # No union splitting needed — type is already concrete from the getter. - return f(pool_any) - end -end - -""" - set_safety_level!(level::Int) -> AdaptiveArrayPool - -Replace the task-local CPU pool (and CUDA pools if CUDA.jl is loaded) -with new pools at the given safety level, preserving cached arrays -and scope state (zero-copy transfer). - -Also updates `POOL_SAFETY_LV[]` so that future `AdaptiveArrayPool()` / -`CuAdaptiveArrayPool()` constructors use the new level. - -## Example -```julia -set_safety_level!(2) # Enable full safety on CPU + all GPU devices -# ... run suspicious code ... -set_safety_level!(0) # Back to zero overhead everywhere -``` - -See also: [`_safety_level`], [`POOL_SAFETY_LV`] -""" -function set_safety_level!(level::Int) - 0 <= level <= 3 || throw(ArgumentError("Safety level must be 0-3; got $level")) - old_pool = get(task_local_storage(), _POOL_KEY, nothing) - if old_pool isa AdaptiveArrayPool - depth = getfield(old_pool, :_current_depth) - depth != 1 && throw( - ArgumentError( - "set_safety_level! cannot be called inside an active @with_pool scope (depth=$depth)" - ) - ) - end - POOL_SAFETY_LV[] = level - new_pool = old_pool === nothing ? _make_pool(level) : _make_pool(level, old_pool::AdaptiveArrayPool) - task_local_storage(_POOL_KEY, new_pool) - # Update CUDA pools if extension is loaded (no-op otherwise) - _set_cuda_safety_level_hook!(level) - return new_pool -end - -# Hook for CUDA extension to override. No-op when CUDA is not loaded. -_set_cuda_safety_level_hook!(::Int) = nothing - # ============================================================================== # CUDA Pool Stubs (overridden by extension when CUDA is loaded) # ============================================================================== diff --git a/src/types.jl b/src/types.jl index 4e9d0c90..a8229a51 100644 --- a/src/types.jl +++ b/src/types.jl @@ -296,69 +296,35 @@ const _TYPE_BITS_MASK = UInt16(0x00FF) # bits 0-7: fixed-slot type bits # ============================================================================== # # Safety is controlled per-pool via the type parameter S in AdaptiveArrayPool{S}. -# S encodes the safety level (0-3), enabling dead-code elimination at compile time. +# S is binary: 0 (off) or 1 (on), enabling dead-code elimination at compile time. # -# 0 = off — zero overhead (default) -# 1 = guard — structural invalidation on rewind (resize + setfield!) -# 2 = full — guard + escape detection + poisoning -# 3 = debug — full + borrow registry (call-site tracking) +# 0 = off — zero overhead (default) +# 1 = on — full runtime checks (invalidation, poisoning, escape detection, borrow tracking) +# +# Int type allows future compile-time check levels (like -O0/-O1/-O2). +# Currently only 0 and 1 are defined. LocalPreferences.toml accepts both +# Bool (true/false) and Int (0/1). # -# DEFAULT_SAFETY_LV sets the initial S for new pools. -# POOL_SAFETY_LV is the runtime Ref read by AdaptiveArrayPool() constructor. -# Use set_safety_level!() to replace the task-local pool at runtime. - using Preferences: @load_preference +_normalize_runtime_check(v::Bool) = Int(v) +_normalize_runtime_check(v::Integer) = clamp(Int(v), 0, 1) + """ - DEFAULT_SAFETY_LV::Int + RUNTIME_CHECK::Int -Compile-time default safety level for new pool instances. +Compile-time constant controlling the runtime safety check level. + +- `0` — off (zero overhead, default) +- `1` — full checks (invalidation, poisoning, escape detection, borrow tracking) Set via `LocalPreferences.toml`: ```toml [AdaptiveArrayPools] -safety_level = 1 +runtime_check = true # or 1 ``` - -Levels: `0` (off, default), `1` (guard), `2` (full), `3` (debug). - -Legacy fallback: reads `pool_checks` preference if `safety_level` is not set -(`true` → 1, `false` → 0). -""" -const DEFAULT_SAFETY_LV = let - sl = @load_preference("safety_level", nothing) - if sl !== nothing - sl::Int - else - # Legacy: pool_checks=true → safety_level=1 - pc = @load_preference("pool_checks", nothing) - pc !== nothing ? (pc::Bool ? 1 : 0) : 0 - end -end - -""" - STATIC_POOL_CHECKS::Bool - -Compile-time constant derived from `DEFAULT_SAFETY_LV > 0`. -Retained for backward compatibility. New code should use `_safety_level(pool) >= N`. -""" -const STATIC_POOL_CHECKS = DEFAULT_SAFETY_LV > 0 - -""" - POOL_SAFETY_LV - -Runtime safety level. Read by `AdaptiveArrayPool()` constructor to determine -the `S` parameter of newly created pools. - -- `0`: Off — zero overhead -- `1`: Guard — structural invalidation on rewind (resize + setfield!) -- `2`: Full — guard + escape detection on scope exit + poisoning -- `3`: Debug — full + borrow registry (acquire call-site tracking) - -Initial value: `DEFAULT_SAFETY_LV` (default `0`). -Use `set_safety_level!()` to change the task-local pool at runtime. """ -const POOL_SAFETY_LV = Ref{Int}(DEFAULT_SAFETY_LV) +const RUNTIME_CHECK = _normalize_runtime_check(@load_preference("runtime_check", 0)) # ============================================================================== # AdaptiveArrayPool @@ -370,11 +336,11 @@ const POOL_SAFETY_LV = Ref{Int}(DEFAULT_SAFETY_LV) Multi-type memory pool with fixed slots for common types and IdDict fallback for others. Zero allocation after warmup. NOT thread-safe - use one pool per Task. -The type parameter `S::Int` encodes the safety level (0-3). Inside `@inline` call chains, -`S` is a compile-time constant — safety checks like `S >= 1` are eliminated by dead-code -elimination when `S = 0`, achieving true zero overhead. +The type parameter `S::Int` encodes the runtime check mode (0 = off, 1 = on). +Inside `@inline` call chains, `S` is a compile-time constant — safety checks are +eliminated by dead-code elimination when `S = 0`, achieving true zero overhead. -See also: [`_safety_level`], [`_make_pool`], [`set_safety_level!`] +See also: [`_runtime_check`], [`_make_pool`], [`RUNTIME_CHECK`] """ mutable struct AdaptiveArrayPool{S} <: AbstractArrayPool # Fixed Slots: common types with zero lookup overhead @@ -395,7 +361,7 @@ mutable struct AdaptiveArrayPool{S} <: AbstractArrayPool _touched_type_masks::Vector{UInt16} # Per-depth: which fixed slots were touched + mode flags _touched_has_others::Vector{Bool} # Per-depth: any non-fixed-slot type touched? - # Borrow registry (POOL_SAFETY_LV >= 3 only) + # Borrow registry (S = 1 only) _pending_callsite::String # "" = no pending; set by macro before acquire _pending_return_site::String # "" = no pending; set by macro before validate _borrow_log::Union{Nothing, IdDict{Any, String}} # vector_obj => callsite string @@ -417,45 +383,46 @@ function AdaptiveArrayPool{S}() where {S} [false], # _touched_has_others: sentinel (no others) "", # _pending_callsite: no pending "", # _pending_return_site: no pending - nothing # _borrow_log: lazily created at LV >= 3 + nothing # _borrow_log: lazily created at S=1 ) end -"""Create pool at the current `POOL_SAFETY_LV[]` level.""" -AdaptiveArrayPool() = _make_pool(POOL_SAFETY_LV[]) +"""Create pool with the default `RUNTIME_CHECK` level.""" +AdaptiveArrayPool() = AdaptiveArrayPool{RUNTIME_CHECK}() """ - _safety_level(pool) -> Int + _runtime_check(pool) -> Bool -Return the safety level of a pool. Compile-time constant for `AdaptiveArrayPool{S}`. +Return whether runtime safety checks are enabled for `pool` (i.e., `S >= 1`). +Compile-time constant for `AdaptiveArrayPool{S}` — dead-code eliminated when `S = 0`. """ -@inline _safety_level(::AdaptiveArrayPool{S}) where {S} = S -@inline _safety_level(::AbstractArrayPool) = POOL_SAFETY_LV[] # fallback +@inline _runtime_check(::AdaptiveArrayPool{0}) = false +@inline _runtime_check(::AdaptiveArrayPool) = true # S >= 1 """ - _make_pool(s::Int) -> AdaptiveArrayPool{s} + _make_pool(level) -> AdaptiveArrayPool -Function barrier: converts runtime `Int` to concrete `AdaptiveArrayPool{S}`. -Levels outside 0-3 are clamped (≤0 → 0, ≥3 → 3). +Function barrier: converts runtime check level to concrete `AdaptiveArrayPool{S}`. +Accepts `Bool` (`true`→1, `false`→0) or `Int` (used directly as S). """ -@noinline function _make_pool(s::Int) - s <= 0 && return AdaptiveArrayPool{0}() - s == 1 && return AdaptiveArrayPool{1}() - s == 2 && return AdaptiveArrayPool{2}() - return AdaptiveArrayPool{3}() +_make_pool(runtime_check::Bool) = _make_pool(Int(runtime_check)) +@noinline function _make_pool(S::Int) + S == 0 && return AdaptiveArrayPool{0}() + return AdaptiveArrayPool{1}() end """ - _make_pool(s::Int, old::AdaptiveArrayPool) -> AdaptiveArrayPool{s} + _make_pool(level, old::AdaptiveArrayPool) -> AdaptiveArrayPool -Create a new pool at safety level `s`, transferring cached arrays and scope state -from `old`. Only reference copies — no memory allocation for the underlying buffers. +Create a new pool, transferring cached arrays and scope state from `old`. +Only reference copies — no memory allocation for the underlying buffers. Transferred: all TypedPool/BitTypedPool slots, `others`, depth & touch tracking. Reset: `_pending_callsite/return_site` (transient macro state), - `_borrow_log` (created fresh when `s >= 3`). + `_borrow_log` (created fresh when S >= 1). """ -@noinline function _make_pool(s::Int, old::AdaptiveArrayPool) +_make_pool(runtime_check::Bool, old::AdaptiveArrayPool) = _make_pool(Int(runtime_check), old) +@noinline function _make_pool(level::Int, old::AdaptiveArrayPool) _new(::Val{S}) where {S} = AdaptiveArrayPool{S}( old.float64, old.float32, old.int64, old.int32, old.complexf64, old.complexf32, old.bool, old.bits, @@ -465,12 +432,10 @@ Reset: `_pending_callsite/return_site` (transient macro state), old._touched_has_others, "", # _pending_callsite: reset "", # _pending_return_site: reset - S >= 3 ? IdDict{Any, String}() : nothing # _borrow_log + S >= 1 ? IdDict{Any, String}() : nothing # _borrow_log ) - s <= 0 && return _new(Val(0)) - s == 1 && return _new(Val(1)) - s == 2 && return _new(Val(2)) - return _new(Val(3)) + level == 0 && return _new(Val(0)) + return _new(Val(1)) end # ============================================================================== @@ -535,12 +500,12 @@ end """ _set_pending_callsite!(pool, msg::String) -Record a pending callsite string for borrow tracking (safety level ≥ 3). +Record a pending callsite string for borrow tracking (S=1). Only sets the callsite if no prior callsite is pending (macro-injected ones take priority). -Compiles to no-op when `S < 3`. +Compiles to no-op when `S=0`. """ @inline function _set_pending_callsite!(pool::AdaptiveArrayPool{S}, msg::String) where {S} - S >= 3 && isempty(pool._pending_callsite) && (pool._pending_callsite = msg) + S >= 1 && isempty(pool._pending_callsite) && (pool._pending_callsite = msg) return nothing end @inline _set_pending_callsite!(::AbstractArrayPool, ::String) = nothing @@ -548,12 +513,12 @@ end """ _maybe_record_borrow!(pool, tp::AbstractTypedPool) -Flush the pending callsite into the borrow log (safety level ≥ 3). +Flush the pending callsite into the borrow log (S=1). Delegates to `_record_borrow_from_pending!` (defined in `debug.jl`). -Compiles to no-op when `S < 3`. +Compiles to no-op when `S=0`. """ @inline function _maybe_record_borrow!(pool::AdaptiveArrayPool{S}, tp::AbstractTypedPool) where {S} - S >= 3 && _record_borrow_from_pending!(pool, tp) + S >= 1 && _record_borrow_from_pending!(pool, tp) return nothing end @inline _maybe_record_borrow!(::AbstractArrayPool, ::AbstractTypedPool) = nothing diff --git a/src/utils.jl b/src/utils.jl index d7e390a8..a64de665 100644 --- a/src/utils.jl +++ b/src/utils.jl @@ -9,12 +9,11 @@ _vector_bytes(v::Vector) = Base.summarysize(v) _count_label(::TypedPool) = "elements" -# Safety level labels for display -_safety_label(::Val{0}) = "off" -_safety_label(::Val{1}) = "guard" -_safety_label(::Val{2}) = "guard+escape" -_safety_label(::Val{3}) = "guard+escape+borrow" -_safety_label(s::Int) = _safety_label(Val(s)) +# Runtime check label for display +_check_label(::Val{0}) = "off" +_check_label(::Val{1}) = "on" +_check_label(s::Bool) = _check_label(Val(s ? 1 : 0)) +_check_label(s::Int) = _check_label(Val(clamp(s, 0, 1))) """ pool_stats(tp::AbstractTypedPool; io::IO=stdout, indent::Int=0, name::String="") @@ -69,10 +68,11 @@ end """ function pool_stats(pool::AdaptiveArrayPool; io::IO = stdout) # Header - S = _safety_level(pool) + check = _runtime_check(pool) + S = check ? 1 : 0 printstyled(io, "AdaptiveArrayPool", bold = true, color = :white) printstyled(io, "{$S}", color = :yellow) - printstyled(io, " (safety=$(_safety_label(S)))", color = :dark_gray) + printstyled(io, " (check=$(_check_label(check)))", color = :dark_gray) println(io) has_content = false @@ -190,8 +190,9 @@ function Base.show(io::IO, pool::AdaptiveArrayPool) total_active[] += tp.n_active end - S = _safety_level(pool) - return print(io, "AdaptiveArrayPool{$S}(safety=$(_safety_label(S)), types=$(n_types[]), slots=$(total_vectors[]), active=$(total_active[]))") + check = _runtime_check(pool) + S = check ? 1 : 0 + return print(io, "AdaptiveArrayPool{$S}(check=$(_check_label(check)), types=$(n_types[]), slots=$(total_vectors[]), active=$(total_active[]))") end # Multi-line show for AdaptiveArrayPool diff --git a/test/cuda/test_allocation.jl b/test/cuda/test_allocation.jl index 974b3f90..3d5a7799 100644 --- a/test/cuda/test_allocation.jl +++ b/test/cuda/test_allocation.jl @@ -242,7 +242,7 @@ end pool = get_task_local_cuda_pool() reset!(pool) - # Wrap in function for proper JIT warmup (_dispatch_pool_scope closure + # Wrap in function for proper JIT warmup (pool binding let-block # needs function boundary to avoid @allocated counting JIT artifacts) function _test_cuda_nd_alloc!() @with_pool :cuda p begin diff --git a/test/cuda/test_cuda_safety.jl b/test/cuda/test_cuda_safety.jl index 7df427f3..2d237008 100644 --- a/test/cuda/test_cuda_safety.jl +++ b/test/cuda/test_cuda_safety.jl @@ -1,31 +1,26 @@ -import AdaptiveArrayPools: PoolRuntimeEscapeError, PoolEscapeError, _safety_level, POOL_DEBUG +import AdaptiveArrayPools: PoolRuntimeEscapeError, PoolEscapeError, _runtime_check, + _validate_pool_return, _lazy_checkpoint!, _lazy_rewind! const _make_cuda_pool = ext._make_cuda_pool # Opaque identity — defeats compile-time escape analysis _cuda_test_leak(x) = x -@testset "CUDA Safety Dispatch (CuAdaptiveArrayPool{S})" begin +@testset "CUDA Safety (CuAdaptiveArrayPool{S}, Binary S=0/1)" begin # ============================================================================== # Type parameterization basics # ============================================================================== - @testset "CuAdaptiveArrayPool{S} construction and _safety_level" begin + @testset "CuAdaptiveArrayPool{S} construction and _runtime_check" begin p0 = _make_cuda_pool(0) p1 = _make_cuda_pool(1) - p2 = _make_cuda_pool(2) - p3 = _make_cuda_pool(3) @test p0 isa CuAdaptiveArrayPool{0} @test p1 isa CuAdaptiveArrayPool{1} - @test p2 isa CuAdaptiveArrayPool{2} - @test p3 isa CuAdaptiveArrayPool{3} - @test _safety_level(p0) == 0 - @test _safety_level(p1) == 1 - @test _safety_level(p2) == 2 - @test _safety_level(p3) == 3 + @test _runtime_check(p0) == false + @test _runtime_check(p1) == true # Borrow fields exist at all levels (required by macro-injected field access) @test hasfield(typeof(p0), :_pending_callsite) @@ -34,10 +29,10 @@ _cuda_test_leak(x) = x end # ============================================================================== - # Level 0: No poisoning, no validation + # S=0: No poisoning, no validation # ============================================================================== - @testset "Level 0: no poisoning on rewind" begin + @testset "S=0: no poisoning on rewind" begin pool = _make_cuda_pool(0) checkpoint!(pool) v = acquire!(pool, Float32, 10) @@ -53,14 +48,38 @@ _cuda_test_leak(x) = x rewind!(pool) end + @testset "S=0: no poisoning (verify data survives rewind)" begin + pool = _make_cuda_pool(0) + checkpoint!(pool) + v = acquire!(pool, Float32, 10) + CUDA.fill!(v, 42.0f0) + rewind!(pool) + + # Data should NOT be poisoned at S=0 + cpu_data = Array(pool.float32.vectors[1]) + @test all(x -> x == 42.0f0, cpu_data[1:10]) + end + + @testset "S=0: no escape detection" begin + pool = _make_cuda_pool(0) + checkpoint!(pool) + try + v = acquire!(pool, Float32, 10) + # Should NOT throw — escape detection requires S=1 + _validate_pool_return(_cuda_test_leak(v), pool) + finally + rewind!(pool) + end + end + # ============================================================================== - # Level 1: Poisoning + structural invalidation (length → 0) + # S=1: Poisoning + structural invalidation + escape detection + borrow tracking # ============================================================================== - # CUDA Level 1 now: poison fill → _resize_to_fit!(vec, 0) + arr_wrappers invalidation + # CUDA S=1: poison fill → _resize_to_fit!(vec, 0) + arr_wrappers invalidation # Backing vector length becomes 0 (GPU memory preserved via maxsize). # Poison data persists in GPU memory and is visible on re-acquire (grow-back). - @testset "Level 1: released vectors have length 0 after rewind" begin + @testset "S=1: released vectors have length 0 after rewind" begin pool = _make_cuda_pool(1) checkpoint!(pool) v = acquire!(pool, Float32, 100) @@ -71,7 +90,7 @@ _cuda_test_leak(x) = x @test length(pool.float32.vectors[1]) == 0 end - @testset "Level 1: Float32 poisoned with NaN on rewind" begin + @testset "S=1: Float32 poisoned with NaN on rewind" begin pool = _make_cuda_pool(1) checkpoint!(pool) v = acquire!(pool, Float32, 10) @@ -88,7 +107,7 @@ _cuda_test_leak(x) = x rewind!(pool) end - @testset "Level 1: Int32 poisoned with typemax on rewind" begin + @testset "S=1: Int32 poisoned with typemax on rewind" begin pool = _make_cuda_pool(1) checkpoint!(pool) v = acquire!(pool, Int32, 8) @@ -102,7 +121,7 @@ _cuda_test_leak(x) = x rewind!(pool) end - @testset "Level 1: ComplexF32 poisoned with NaN on rewind" begin + @testset "S=1: ComplexF32 poisoned with NaN on rewind" begin pool = _make_cuda_pool(1) checkpoint!(pool) v = acquire!(pool, ComplexF32, 8) @@ -115,7 +134,7 @@ _cuda_test_leak(x) = x rewind!(pool) end - @testset "Level 1: Bool poisoned with true on rewind" begin + @testset "S=1: Bool poisoned with true on rewind" begin pool = _make_cuda_pool(1) checkpoint!(pool) v = acquire!(pool, Bool, 16) @@ -128,7 +147,7 @@ _cuda_test_leak(x) = x rewind!(pool) end - @testset "Level 1: Float16 poisoned with NaN on rewind" begin + @testset "S=1: Float16 poisoned with NaN on rewind" begin pool = _make_cuda_pool(1) checkpoint!(pool) v = acquire!(pool, Float16, 10) @@ -141,7 +160,7 @@ _cuda_test_leak(x) = x rewind!(pool) end - @testset "Level 1: arr_wrappers invalidated on poisoned rewind" begin + @testset "S=1: arr_wrappers invalidated on rewind" begin pool = _make_cuda_pool(1) checkpoint!(pool) v = acquire!(pool, Float32, 10) @@ -160,102 +179,59 @@ _cuda_test_leak(x) = x end end - @testset "Level 1: no escape detection" begin - # Level 1 should NOT throw on escape (that's Level 2+) - pool = _make_cuda_pool(1) - result = begin - checkpoint!(pool) - v = acquire!(pool, Float32, 10) - rewind!(pool) - v # "escaping" — should not throw at Level 1 - end - @test result isa CuArray - end - # ============================================================================== - # Level 0: Verify no poisoning + # S=1: Escape detection # ============================================================================== - @testset "Level 0: no poisoning (verify data survives rewind)" begin - pool = _make_cuda_pool(0) - checkpoint!(pool) - v = acquire!(pool, Float32, 10) - CUDA.fill!(v, 42.0f0) - rewind!(pool) - - # Data should NOT be poisoned at Level 0 - cpu_data = Array(pool.float32.vectors[1]) - @test all(x -> x == 42.0f0, cpu_data[1:10]) - end - - # ============================================================================== - # Level 2: Escape detection - # ============================================================================== - - @testset "Level 2: escape detection catches CuArray leak" begin - pool = _make_cuda_pool(2) + @testset "S=1: escape detection catches CuArray leak" begin + pool = _make_cuda_pool(1) @test_throws PoolRuntimeEscapeError begin checkpoint!(pool) try v = acquire!(pool, Float32, 10) - # Simulate what _validate_pool_return does - AdaptiveArrayPools._validate_pool_return(_cuda_test_leak(v), pool) + _validate_pool_return(_cuda_test_leak(v), pool) finally rewind!(pool) end end end - @testset "Level 2: safe scalar return does not throw" begin - pool = _make_cuda_pool(2) + @testset "S=1: safe scalar return does not throw" begin + pool = _make_cuda_pool(1) checkpoint!(pool) try v = acquire!(pool, Float32, 10) CUDA.fill!(v, 3.0f0) result = sum(Array(v)) # scalar — safe - AdaptiveArrayPools._validate_pool_return(result, pool) + _validate_pool_return(result, pool) @test result == 30.0f0 finally rewind!(pool) end end - @testset "Level 2: escape detection with Tuple containing CuArray" begin - pool = _make_cuda_pool(2) + @testset "S=1: escape detection with Tuple containing CuArray" begin + pool = _make_cuda_pool(1) @test_throws PoolRuntimeEscapeError begin checkpoint!(pool) try v = acquire!(pool, Float32, 10) val = (42, _cuda_test_leak(v)) - AdaptiveArrayPools._validate_pool_return(val, pool) + _validate_pool_return(val, pool) finally rewind!(pool) end end end - @testset "Level 2: escape detection with Dict containing CuArray" begin - pool = _make_cuda_pool(2) + @testset "S=1: escape detection with Dict containing CuArray" begin + pool = _make_cuda_pool(1) @test_throws PoolRuntimeEscapeError begin checkpoint!(pool) try v = acquire!(pool, Float32, 10) val = Dict(:data => _cuda_test_leak(v)) - AdaptiveArrayPools._validate_pool_return(val, pool) - finally - rewind!(pool) - end - end - end - - @testset "Level 0 and 1: no escape detection" begin - for lv in (0, 1) - pool = _make_cuda_pool(lv) - checkpoint!(pool) - try - v = acquire!(pool, Float32, 10) - # Should NOT throw — escape detection requires Level 2+ - AdaptiveArrayPools._validate_pool_return(_cuda_test_leak(v), pool) + _validate_pool_return(val, pool) finally rewind!(pool) end @@ -263,29 +239,29 @@ _cuda_test_leak(x) = x end # ============================================================================== - # Level 3: Borrow tracking + # S=1: Borrow tracking # ============================================================================== - @testset "Level 3: borrow fields functional" begin - pool = _make_cuda_pool(3) + @testset "S=1: borrow fields functional" begin + pool = _make_cuda_pool(1) @test pool._pending_callsite == "" @test pool._pending_return_site == "" @test pool._borrow_log === nothing # lazily created end - @testset "Level 3: _set_pending_callsite! works" begin - pool = _make_cuda_pool(3) + @testset "S=1: _set_pending_callsite! works" begin + pool = _make_cuda_pool(1) AdaptiveArrayPools._set_pending_callsite!(pool, "test.jl:42\nacquire!(pool, Float32, 10)") @test pool._pending_callsite == "test.jl:42\nacquire!(pool, Float32, 10)" - # At Level 0, should be no-op + # At S=0, should be no-op pool0 = _make_cuda_pool(0) AdaptiveArrayPools._set_pending_callsite!(pool0, "should not be set") @test pool0._pending_callsite == "" end - @testset "Level 3: _maybe_record_borrow! records callsite" begin - pool = _make_cuda_pool(3) + @testset "S=1: _maybe_record_borrow! records callsite" begin + pool = _make_cuda_pool(1) checkpoint!(pool) tp = get_typed_pool!(pool, Float32) @@ -300,43 +276,21 @@ _cuda_test_leak(x) = x rewind!(pool) end - # ============================================================================== - # set_safety_level! — all-device replacement - # ============================================================================== - - @testset "set_safety_level! replaces pool with state preservation" begin - # Get current pool (creates one at default safety level) - pool = get_task_local_cuda_pool() - reset!(pool) - - # Populate with some data + @testset "S=0: does not create borrow log on CUDA" begin + pool = _make_cuda_pool(0) checkpoint!(pool) - v = acquire!(pool, Float32, 100) - CUDA.fill!(v, 1.0f0) + _ = acquire!(pool, Float32, 10) + @test pool._borrow_log === nothing rewind!(pool) - - # Change safety level - set_safety_level!(2) - new_pool = get_task_local_cuda_pool() - - @test new_pool isa CuAdaptiveArrayPool{2} - @test _safety_level(new_pool) == 2 - # Cached vectors should be preserved (same object reference) - @test new_pool.float32.vectors[1] === pool.float32.vectors[1] - - # Restore - set_safety_level!(0) - @test get_task_local_cuda_pool() isa CuAdaptiveArrayPool{0} end - @testset "set_safety_level! rejects inside active scope" begin - pool = get_task_local_cuda_pool() + @testset "S=1: creates borrow log on CUDA acquire" begin + pool = _make_cuda_pool(1) checkpoint!(pool) - try - @test_throws ArgumentError set_safety_level!(2) - finally - rewind!(pool) - end + _ = acquire!(pool, Float32, 10) + @test pool._borrow_log !== nothing + @test pool._borrow_log isa IdDict + rewind!(pool) end # ============================================================================== @@ -383,7 +337,7 @@ _cuda_test_leak(x) = x # ============================================================================== @testset "reset! clears borrow tracking state" begin - pool = _make_cuda_pool(3) + pool = _make_cuda_pool(1) pool._pending_callsite = "test" pool._pending_return_site = "test" pool._borrow_log = IdDict{Any, String}() @@ -395,44 +349,6 @@ _cuda_test_leak(x) = x @test pool._borrow_log === nothing end - # ============================================================================== - # Display includes {S} and safety label - # ============================================================================== - - @testset "show includes {S} and safety label" begin - pool = _make_cuda_pool(2) - s = sprint(show, pool) - @test occursin("{2}", s) - @test occursin("safety=full", s) - - pool0 = _make_cuda_pool(0) - s0 = sprint(show, pool0) - @test occursin("{0}", s0) - @test occursin("safety=off", s0) - end - - # ============================================================================== - # POOL_DEBUG backward compat with CUDA - # ============================================================================== - - @testset "POOL_DEBUG backward compat triggers CUDA escape detection" begin - old_debug = POOL_DEBUG[] - - POOL_DEBUG[] = true - pool = _make_cuda_pool(0) # Safety off, but POOL_DEBUG overrides - @test_throws PoolRuntimeEscapeError begin - checkpoint!(pool) - try - v = acquire!(pool, Float32, 10) - AdaptiveArrayPools._validate_pool_return(_cuda_test_leak(v), pool) - finally - rewind!(pool) - end - end - - POOL_DEBUG[] = old_debug - end - # ============================================================================== # Fallback types (pool.others) poisoning # ============================================================================== @@ -456,33 +372,19 @@ _cuda_test_leak(x) = x end # ============================================================================== - # @with_pool :cuda integration with safety + # Display includes {S} and check label # ============================================================================== - @testset "@with_pool :cuda with escape detection" begin - old_debug = POOL_DEBUG[] - POOL_DEBUG[] = true # Use POOL_DEBUG to trigger on any safety level + @testset "show includes {S} and check label" begin + pool1 = _make_cuda_pool(1) + s1 = sprint(show, pool1) + @test occursin("{1}", s1) + @test occursin("check=on", s1) - @test_throws PoolRuntimeEscapeError @with_pool :cuda pool begin - v = acquire!(pool, Float32, 10) - _cuda_test_leak(v) - end - - POOL_DEBUG[] = old_debug - end - - @testset "@with_pool :cuda safe return" begin - old_debug = POOL_DEBUG[] - POOL_DEBUG[] = true - - result = @with_pool :cuda pool begin - v = acquire!(pool, Float32, 10) - CUDA.fill!(v, 3.0f0) - sum(Array(v)) # scalar return — safe - end - @test result == 30.0f0 - - POOL_DEBUG[] = old_debug + pool0 = _make_cuda_pool(0) + s0 = sprint(show, pool0) + @test occursin("{0}", s0) + @test occursin("check=off", s0) end # ============================================================================== @@ -513,108 +415,61 @@ _cuda_test_leak(x) = x end # ============================================================================== - # @with_pool :cuda at native Level 2 (no POOL_DEBUG hack) + # S=1 escape detection via direct checkpoint/validate/rewind + # (replaces old set_safety_level! + @with_pool tests) # ============================================================================== - @testset "@with_pool :cuda Level 2 escape detection (native S=2)" begin - set_safety_level!(2) - - @test_throws PoolRuntimeEscapeError @with_pool :cuda pool begin + @testset "Pool{1} escape detection via direct validate" begin + pool = _make_cuda_pool(1) + checkpoint!(pool) + err = try v = acquire!(pool, Float32, 10) - _cuda_test_leak(v) + _validate_pool_return(_cuda_test_leak(v), pool) + nothing + catch e + e + finally + rewind!(pool) end - set_safety_level!(0) + @test err isa PoolRuntimeEscapeError end - @testset "@with_pool :cuda Level 2 safe return (native S=2)" begin - set_safety_level!(2) - - result = @with_pool :cuda pool begin - v = acquire!(pool, Float32, 10) - CUDA.fill!(v, 5.0f0) - sum(Array(v)) - end + @testset "Pool{1} safe scalar via direct validate" begin + pool = _make_cuda_pool(1) + checkpoint!(pool) + v = acquire!(pool, Float32, 10) + CUDA.fill!(v, 5.0f0) + result = sum(Array(v)) + _validate_pool_return(result, pool) + rewind!(pool) @test result == 50.0f0 - - set_safety_level!(0) - end - - @testset "@with_pool :cuda Level 1 no escape detection (native S=1)" begin - set_safety_level!(1) - - # Level 1 should NOT trigger escape detection - result = @with_pool :cuda pool begin - v = acquire!(pool, Float32, 10) - _cuda_test_leak(v) - end - @test result isa CuArray - - set_safety_level!(0) end # ============================================================================== - # Level 3 borrow tracking via macro path + # S=1 borrow tracking: callsite in escape error # ============================================================================== - @testset "@with_pool :cuda Level 3 escape error includes callsite" begin - set_safety_level!(3) - - err = try - @with_pool :cuda pool begin - v = acquire!(pool, Float32, 10) - _cuda_test_leak(v) - end - nothing - catch e - e - end - - @test err isa PoolRuntimeEscapeError - @test err.callsite !== nothing - @test contains(err.callsite, ":") # "file:line" format - - set_safety_level!(0) - end + @testset "Pool{1} escape error includes callsite when set" begin + pool = _make_cuda_pool(1) + checkpoint!(pool) - @testset "@with_pool :cuda Level 3 callsite includes expression text" begin - set_safety_level!(3) + # Manually set callsite (normally macro-injected) + pool._pending_callsite = "test_cuda.jl:42\nacquire!(pool, Float32, 10)" + v = acquire!(pool, Float32, 10) err = try - @with_pool :cuda pool begin - v = zeros!(pool, Float32, 10) - _cuda_test_leak(v) - end + _validate_pool_return(_cuda_test_leak(v), pool) nothing catch e e end + rewind!(pool) @test err isa PoolRuntimeEscapeError @test err.callsite !== nothing - @test contains(err.callsite, "\n") - @test contains(err.callsite, "zeros!(pool, Float32, 10)") - - set_safety_level!(0) - end - - @testset "LV<3 does not create borrow log on CUDA" begin - for lv in (0, 1, 2) - pool = _make_cuda_pool(lv) - checkpoint!(pool) - _ = acquire!(pool, Float32, 10) - @test pool._borrow_log === nothing - rewind!(pool) - end - end - - @testset "LV=3 creates borrow log on CUDA acquire" begin - pool = _make_cuda_pool(3) - checkpoint!(pool) - _ = acquire!(pool, Float32, 10) - @test pool._borrow_log !== nothing - @test pool._borrow_log isa IdDict - rewind!(pool) + @test contains(err.callsite, "test_cuda.jl:42") + @test contains(err.callsite, "acquire!(pool, Float32, 10)") end # ============================================================================== @@ -622,7 +477,6 @@ _cuda_test_leak(x) = x # ============================================================================== @testset "showerror: CuArray escape error message format" begin - # LV≥2 without callsite → "Tip: set LV=3" err = PoolRuntimeEscapeError("CuArray{Float32, 1}", "Float32", nothing, nothing) io = IOBuffer() showerror(io, err) @@ -631,12 +485,10 @@ _cuda_test_leak(x) = x @test contains(msg, "PoolEscapeError") @test contains(msg, "CuArray{Float32, 1}") @test contains(msg, "Float32") - @test contains(msg, "POOL_SAFETY_LV ≥ 2") - @test contains(msg, "Tip:") - @test contains(msg, "POOL_SAFETY_LV[] = 3") + @test contains(msg, "RUNTIME_CHECK") end - @testset "showerror: CuArray with callsite (LV≥3)" begin + @testset "showerror: CuArray with callsite" begin err = PoolRuntimeEscapeError( "CuArray{Float32, 1}", "Float32", "test_cuda.jl:42\nacquire!(pool, Float32, 10)", nothing @@ -648,52 +500,36 @@ _cuda_test_leak(x) = x @test contains(msg, "acquired at") @test contains(msg, "test_cuda.jl:42") @test contains(msg, "acquire!(pool, Float32, 10)") - @test contains(msg, "POOL_SAFETY_LV ≥ 3") - @test !contains(msg, "Tip:") # No tip when callsite is present end # ============================================================================== # Function form: @with_pool :cuda pool function ... + # (Compile-time only — no runtime escape detection test via @with_pool, + # because RUNTIME_CHECK is a compile-time const and @with_pool type-asserts + # to Pool{RUNTIME_CHECK}. Use direct _make_pool(1) + validate for runtime tests.) # ============================================================================== - @testset "Function form: escape detection with explicit return" begin - set_safety_level!(2) - - @with_pool :cuda pool function _cuda_test_return_escape() + @testset "Function form: compile-time escape detection" begin + @test_throws PoolEscapeError @macroexpand @with_pool :cuda pool function _cuda_test_escape_fn() v = acquire!(pool, Float32, 10) - return _cuda_test_leak(v) + return v # direct escape end - - @test_throws PoolRuntimeEscapeError _cuda_test_return_escape() - - set_safety_level!(0) end - @testset "Function form: safe scalar return passes" begin - set_safety_level!(2) - - @with_pool :cuda pool function _cuda_test_safe_return() + @testset "Function form: safe scalar return compiles" begin + ex = @macroexpand @with_pool :cuda pool function _cuda_test_safe_fn() v = acquire!(pool, Float32, 5) - CUDA.fill!(v, 4.0f0) return sum(Array(v)) end - - @test _cuda_test_safe_return() == 20.0f0 - - set_safety_level!(0) + @test ex isa Expr end - @testset "Function form: bare return (nothing) passes" begin - set_safety_level!(2) - - @with_pool :cuda pool function _cuda_test_bare_return() + @testset "Function form: bare return compiles" begin + ex = @macroexpand @with_pool :cuda pool function _cuda_test_bare_fn() _ = acquire!(pool, Float32, 10) return end - - @test _cuda_test_bare_return() === nothing - - set_safety_level!(0) + @test ex isa Expr end -end # CUDA Safety Dispatch +end # CUDA Safety diff --git a/test/test_allocation.jl b/test/test_allocation.jl index 949b8e9f..d43a17d7 100644 --- a/test/test_allocation.jl +++ b/test/test_allocation.jl @@ -20,10 +20,8 @@ end @testset "zero allocation on reuse" begin - # Disable safety invalidation: rewind-time resize!/setfield! forces cache misses - # (new SubArray views on legacy, new BitArray wrappers), breaking zero-alloc invariant. - old_safety = POOL_SAFETY_LV[] - set_safety_level!(0) + # RUNTIME_CHECK=0 (default) → Pool{0}, no invalidation overhead. + # Zero-alloc invariant holds because rewind-time resize!/setfield! is dead-code eliminated. # First call: JIT + initial cache miss (pool arrays + N-way bitarray cache) alloc1 = @allocated foo() @@ -40,6 +38,4 @@ end alloc3 = @allocated foo() @test alloc2 == 0 @test alloc3 == 0 - - set_safety_level!(old_safety) end diff --git a/test/test_backend_macro_expansion.jl b/test/test_backend_macro_expansion.jl index ad292cc7..4440761f 100644 --- a/test/test_backend_macro_expansion.jl +++ b/test/test_backend_macro_expansion.jl @@ -490,8 +490,8 @@ end body_str = string(expr.args[2]) - # @with_pool has POOL_DEBUG check but NOT MAYBE_POOLING runtime toggle - @test occursin("_validate_pool_return", body_str) # POOL_DEBUG present + # @with_pool has _runtime_check gate but NOT MAYBE_POOLING runtime toggle + @test occursin("_validate_pool_return", body_str) # runtime check present @test !occursin("DisabledPool", body_str) # No MAYBE_POOLING branch @test occursin("_get_pool_for_backend", body_str) end diff --git a/test/test_basic.jl b/test/test_basic.jl index 8ad42a28..65998c2c 100644 --- a/test/test_basic.jl +++ b/test/test_basic.jl @@ -54,7 +54,7 @@ end @testset "Slot reuse with resize via _claim_slot!" begin - # Use S=0 pool: safety level is baked into type, so we need an explicit + # Use S=0 pool: runtime check level is baked into type, so we need an explicit # AdaptiveArrayPool{0} to test capacity preservation without invalidation. pool = AdaptiveArrayPool{0}() diff --git a/test/test_borrow_registry.jl b/test/test_borrow_registry.jl index b180141a..e1e166d6 100644 --- a/test/test_borrow_registry.jl +++ b/test/test_borrow_registry.jl @@ -1,54 +1,52 @@ import AdaptiveArrayPools: _validate_pool_return, _lookup_borrow_callsite, - PoolRuntimeEscapeError, Bit + PoolRuntimeEscapeError, Bit, _make_pool, _lazy_checkpoint!, _lazy_rewind! _test_leak(x) = x -@testset "Borrow Registry (POOL_SAFETY_LV=3)" begin +@testset "Borrow Registry (RUNTIME_CHECK)" begin # ============================================================================== - # Basic recording: LV=3 macro path → callsite in escape error + # Basic recording: S=1 direct path → callsite in escape error # ============================================================================== - @testset "Macro path: escape error includes callsite" begin - old_lv = POOL_SAFETY_LV[] - set_safety_level!(3) # creates Pool{3} in task-local storage + @testset "Direct path: escape error includes callsite" begin + pool = _make_pool(true) + _lazy_checkpoint!(pool) err = try - @with_pool pool begin - v = acquire!(pool, Float64, 10) - _test_leak(v) - end + pool._pending_callsite = "test_borrow:1\nv = acquire!(pool, Float64, 10)" + v = acquire!(pool, Float64, 10) + _validate_pool_return(_test_leak(v), pool) nothing catch e e + finally + _lazy_rewind!(pool) end @test err isa PoolRuntimeEscapeError @test err.callsite !== nothing @test contains(err.callsite, ":") # "file:line" format - - set_safety_level!(old_lv) end - @testset "Macro path: unsafe_acquire! escape includes callsite" begin - old_lv = POOL_SAFETY_LV[] - set_safety_level!(3) + @testset "Direct path: unsafe_acquire! escape includes callsite" begin + pool = _make_pool(true) + _lazy_checkpoint!(pool) err = try - @with_pool pool begin - v = unsafe_acquire!(pool, Float64, 10) - _test_leak(v) - end + pool._pending_callsite = "test_borrow:2\nv = unsafe_acquire!(pool, Float64, 10)" + v = unsafe_acquire!(pool, Float64, 10) + _validate_pool_return(_test_leak(v), pool) nothing catch e e + finally + _lazy_rewind!(pool) end @test err isa PoolRuntimeEscapeError @test err.callsite !== nothing @test contains(err.callsite, ":") - - set_safety_level!(old_lv) end # ============================================================================== @@ -56,11 +54,8 @@ _test_leak(x) = x # ============================================================================== @testset "Direct acquire! shows generic callsite label" begin - old_lv = POOL_SAFETY_LV[] - set_safety_level!(3) - - pool = AdaptiveArrayPool() - checkpoint!(pool) + pool = _make_pool(true) + _lazy_checkpoint!(pool) v = acquire!(pool, Float64, 10) err = try @@ -73,16 +68,12 @@ _test_leak(x) = x @test err isa PoolRuntimeEscapeError @test err.callsite == "" - rewind!(pool) - set_safety_level!(old_lv) + _lazy_rewind!(pool) end @testset "Direct unsafe_acquire! shows generic callsite label" begin - old_lv = POOL_SAFETY_LV[] - set_safety_level!(3) - - pool = AdaptiveArrayPool() - checkpoint!(pool) + pool = _make_pool(true) + _lazy_checkpoint!(pool) v = unsafe_acquire!(pool, Float64, 10) err = try @@ -95,47 +86,46 @@ _test_leak(x) = x @test err isa PoolRuntimeEscapeError @test err.callsite == "" - rewind!(pool) - set_safety_level!(old_lv) + _lazy_rewind!(pool) end # ============================================================================== - # Convenience functions via macro → callsite + # Convenience functions via direct path → callsite # ============================================================================== - @testset "Macro path: zeros! escape includes callsite" begin - old_lv = POOL_SAFETY_LV[] - set_safety_level!(3) + @testset "Direct path: zeros! escape includes callsite" begin + pool = _make_pool(true) + _lazy_checkpoint!(pool) err = try - @with_pool pool begin - v = zeros!(pool, Float64, 10) - _test_leak(v) - end + pool._pending_callsite = "test_borrow:3\nv = zeros!(pool, Float64, 10)" + v = zeros!(pool, Float64, 10) + _validate_pool_return(_test_leak(v), pool) nothing catch e e + finally + _lazy_rewind!(pool) end @test err isa PoolRuntimeEscapeError @test err.callsite !== nothing @test contains(err.callsite, ":") - - set_safety_level!(old_lv) end - @testset "Macro path: callsite includes expression text" begin - old_lv = POOL_SAFETY_LV[] - set_safety_level!(3) + @testset "Direct path: callsite includes expression text" begin + pool = _make_pool(true) + _lazy_checkpoint!(pool) err = try - @with_pool pool begin - v = zeros!(pool, Float64, 10) - _test_leak(v) - end + pool._pending_callsite = "test_borrow:4\nzeros!(pool, Float64, 10)" + v = zeros!(pool, Float64, 10) + _validate_pool_return(_test_leak(v), pool) nothing catch e e + finally + _lazy_rewind!(pool) end @test err isa PoolRuntimeEscapeError @@ -143,16 +133,11 @@ _test_leak(x) = x # Callsite should contain expression text after \n @test contains(err.callsite, "\n") @test contains(err.callsite, "zeros!(pool, Float64, 10)") - - set_safety_level!(old_lv) end @testset "Direct zeros! shows generic callsite label" begin - old_lv = POOL_SAFETY_LV[] - set_safety_level!(3) - - pool = AdaptiveArrayPool() - checkpoint!(pool) + pool = _make_pool(true) + _lazy_checkpoint!(pool) v = zeros!(pool, Float64, 10) err = try @@ -165,70 +150,56 @@ _test_leak(x) = x @test err isa PoolRuntimeEscapeError @test err.callsite == "" - rewind!(pool) - set_safety_level!(old_lv) + _lazy_rewind!(pool) end # ============================================================================== # BitArray path → callsite # ============================================================================== - @testset "Macro path: BitArray acquire escape includes callsite" begin - old_lv = POOL_SAFETY_LV[] - set_safety_level!(3) + @testset "Direct path: BitArray acquire escape includes callsite" begin + pool = _make_pool(true) + _lazy_checkpoint!(pool) err = try - @with_pool pool begin - v = acquire!(pool, Bit, 100) - _test_leak(v) - end + pool._pending_callsite = "test_borrow:5\nv = acquire!(pool, Bit, 100)" + v = acquire!(pool, Bit, 100) + _validate_pool_return(_test_leak(v), pool) nothing catch e e + finally + _lazy_rewind!(pool) end @test err isa PoolRuntimeEscapeError @test err.callsite !== nothing @test contains(err.callsite, ":") - - set_safety_level!(old_lv) end # ============================================================================== - # LV<3: no borrow log overhead + # S=0: no borrow log overhead # ============================================================================== - @testset "LV<3 does not create borrow log" begin - for lv in (0, 1, 2) - old_lv = POOL_SAFETY_LV[] - set_safety_level!(lv) - - pool = AdaptiveArrayPool() - checkpoint!(pool) - _ = acquire!(pool, Float64, 10) - @test pool._borrow_log === nothing - rewind!(pool) - - set_safety_level!(old_lv) - end + @testset "S=0 does not create borrow log" begin + pool = _make_pool(false) + _lazy_checkpoint!(pool) + _ = acquire!(pool, Float64, 10) + @test pool._borrow_log === nothing + _lazy_rewind!(pool) end # ============================================================================== - # LV=3: borrow log IS created + # S=1: borrow log IS created # ============================================================================== - @testset "LV=3 creates borrow log on acquire" begin - old_lv = POOL_SAFETY_LV[] - set_safety_level!(3) - - pool = AdaptiveArrayPool() - checkpoint!(pool) + @testset "S=1 creates borrow log on acquire" begin + pool = _make_pool(true) + _lazy_checkpoint!(pool) _ = acquire!(pool, Float64, 10) @test pool._borrow_log !== nothing @test pool._borrow_log isa IdDict - rewind!(pool) - - set_safety_level!(old_lv) + _lazy_rewind!(pool) end # ============================================================================== @@ -236,26 +207,21 @@ _test_leak(x) = x # ============================================================================== @testset "reset! clears borrow log and pending callsite" begin - old_lv = POOL_SAFETY_LV[] - set_safety_level!(3) - - pool = AdaptiveArrayPool() - checkpoint!(pool) + pool = _make_pool(true) + _lazy_checkpoint!(pool) _ = acquire!(pool, Float64, 10) @test pool._borrow_log !== nothing reset!(pool) @test pool._borrow_log === nothing @test pool._pending_callsite == "" - - set_safety_level!(old_lv) end # ============================================================================== # Error message format: showerror output # ============================================================================== - @testset "showerror: 'acquired at' shown when callsite present (LV≥3)" begin + @testset "showerror: 'acquired at' shown when callsite present (S=1)" begin err = PoolRuntimeEscapeError("SubArray{Float64, 1}", "Float64", "test.jl:42", nothing) io = IOBuffer() showerror(io, err) @@ -263,7 +229,7 @@ _test_leak(x) = x @test contains(msg, "acquired at") @test contains(msg, "test.jl:42") - @test contains(msg, "POOL_SAFETY_LV ≥ 3") + @test contains(msg, "RUNTIME_CHECK >= 1") @test !contains(msg, "Tip:") end @@ -297,16 +263,14 @@ _test_leak(x) = x @test contains(msg, "acquire!(pool, Float64, 5)") end - @testset "showerror: 'Tip: set LV=3' shown when no callsite (LV=2)" begin + @testset "showerror: no callsite still works" begin err = PoolRuntimeEscapeError("SubArray{Float64, 1}", "Float64", nothing, nothing) io = IOBuffer() showerror(io, err) msg = String(take!(io)) @test !contains(msg, "acquired at") - @test contains(msg, "POOL_SAFETY_LV ≥ 2") - @test contains(msg, "Tip:") - @test contains(msg, "POOL_SAFETY_LV[] = 3") + @test contains(msg, "RUNTIME_CHECK >= 1") end # ============================================================================== @@ -314,11 +278,8 @@ _test_leak(x) = x # ============================================================================== @testset "Multiple types record independent callsites" begin - old_lv = POOL_SAFETY_LV[] - set_safety_level!(3) - - pool = AdaptiveArrayPool() - checkpoint!(pool) + pool = _make_pool(true) + _lazy_checkpoint!(pool) v_f64 = acquire!(pool, Float64, 10) v_i32 = acquire!(pool, Int32, 5) @@ -336,8 +297,7 @@ _test_leak(x) = x @test cs_f64 == "" @test cs_i32 == "" - rewind!(pool) - set_safety_level!(old_lv) + _lazy_rewind!(pool) end # ============================================================================== @@ -345,86 +305,81 @@ _test_leak(x) = x # ============================================================================== @testset "Function form: explicit return triggers escape detection" begin - old_lv = POOL_SAFETY_LV[] - set_safety_level!(2) # creates Pool{2} for escape detection + pool = _make_pool(true) + _lazy_checkpoint!(pool) - # Function with explicit return of pool-backed array should throw - @with_pool pool function _test_return_escape() + err = try v = acquire!(pool, Float64, 10) - return _test_leak(v) + _validate_pool_return(_test_leak(v), pool) + nothing + catch e + e + finally + _lazy_rewind!(pool) end - @test_throws PoolRuntimeEscapeError _test_return_escape() - - set_safety_level!(old_lv) + @test err isa PoolRuntimeEscapeError end @testset "Function form: safe return passes validation" begin - old_lv = POOL_SAFETY_LV[] - set_safety_level!(2) + pool = _make_pool(true) + _lazy_checkpoint!(pool) - @with_pool pool function _test_safe_return() - v = acquire!(pool, Float64, 5) - v .= 3.0 - return sum(v) # scalar — safe - end + v = acquire!(pool, Float64, 5) + v .= 3.0 + result = sum(v) # scalar — safe + _validate_pool_return(result, pool) - @test _test_safe_return() == 15.0 + _lazy_rewind!(pool) - set_safety_level!(old_lv) + @test result == 15.0 end @testset "Function form: bare return (nothing) passes" begin - old_lv = POOL_SAFETY_LV[] - set_safety_level!(2) + pool = _make_pool(true) + _lazy_checkpoint!(pool) - @with_pool pool function _test_bare_return() - _ = acquire!(pool, Float64, 10) - return - end - - @test _test_bare_return() === nothing + _ = acquire!(pool, Float64, 10) + _validate_pool_return(nothing, pool) - set_safety_level!(old_lv) + _lazy_rewind!(pool) end - @testset "Function form: return with callsite at LV=3" begin - old_lv = POOL_SAFETY_LV[] - set_safety_level!(3) - - @with_pool pool function _test_return_callsite() - v = acquire!(pool, Float64, 10) - return _test_leak(v) - end + @testset "Function form: return with callsite at S=1" begin + pool = _make_pool(true) + _lazy_checkpoint!(pool) + pool._pending_callsite = "test_borrow:6\nv = acquire!(pool, Float64, 10)" err = try - _test_return_callsite() + v = acquire!(pool, Float64, 10) + _validate_pool_return(_test_leak(v), pool) nothing catch e e + finally + _lazy_rewind!(pool) end @test err isa PoolRuntimeEscapeError @test err.callsite !== nothing @test contains(err.callsite, ":") - - set_safety_level!(old_lv) end @testset "Block form: return in enclosing function triggers validation" begin - old_lv = POOL_SAFETY_LV[] - set_safety_level!(2) - - function _test_block_return_escape() - @with_pool pool begin - v = acquire!(pool, Float64, 10) - return _test_leak(v) - end - end + pool = _make_pool(true) + _lazy_checkpoint!(pool) - @test_throws PoolRuntimeEscapeError _test_block_return_escape() + err = try + v = acquire!(pool, Float64, 10) + _validate_pool_return(_test_leak(v), pool) + nothing + catch e + e + finally + _lazy_rewind!(pool) + end - set_safety_level!(old_lv) + @test err isa PoolRuntimeEscapeError end end diff --git a/test/test_debug.jl b/test/test_debug.jl index 78af99fe..acd97032 100644 --- a/test/test_debug.jl +++ b/test/test_debug.jl @@ -1,57 +1,9 @@ import AdaptiveArrayPools: _validate_pool_return, _check_bitchunks_overlap, _eltype_may_contain_arrays, - PoolRuntimeEscapeError, _poison_value, _shorten_location + PoolRuntimeEscapeError, _poison_value, _shorten_location, + _make_pool, _lazy_checkpoint!, _lazy_rewind! _test_leak(x) = x # opaque to compile-time escape checker (only identity() is transparent) -@testset "POOL_DEBUG and Safety Validation" begin - - # ============================================================================== - # POOL_DEBUG flag toggle - # ============================================================================== - - @testset "POOL_DEBUG flag" begin - old_debug = POOL_DEBUG[] - - # Default is false - POOL_DEBUG[] = false - - # When debug is off, no validation happens even if SubArray escapes - result = @with_pool pool begin - v = acquire!(pool, Float64, 10) - _test_leak(v) # opaque to compile-time checker; runtime LV<2 won't catch - end - @test result isa SubArray # No error when debug is off - - POOL_DEBUG[] = old_debug - end - - @testset "POOL_DEBUG with safety violation" begin - old_debug = POOL_DEBUG[] - POOL_DEBUG[] = true - - # Should throw error when returning SubArray with debug on - @test_throws PoolRuntimeEscapeError @with_pool pool begin - v = acquire!(pool, Float64, 10) - _test_leak(v) # opaque to compile-time checker; caught by runtime LV2 - end - - # Safe returns should work fine - result = @with_pool pool begin - v = acquire!(pool, Float64, 10) - v .= 1.0 - sum(v) # Safe: returning scalar - end - @test result == 10.0 - - # Returning a copy is also safe - result = @with_pool pool begin - v = acquire!(pool, Float64, 5) - v .= 2.0 - collect(v) # Safe: returning a copy - end - @test result == [2.0, 2.0, 2.0, 2.0, 2.0] - - POOL_DEBUG[] = old_debug - end +@testset "Safety Validation" begin # ============================================================================== # _validate_pool_return — direct tests @@ -199,39 +151,58 @@ _test_leak(x) = x # opaque to compile-time escape checker (only identity() is t rewind!(pool) end - @testset "POOL_DEBUG with N-D arrays" begin - old_debug = POOL_DEBUG[] - POOL_DEBUG[] = true - + @testset "Pool{1} escape detection with N-D arrays" begin # N-D ReshapedArray should throw error when returned - @test_throws PoolRuntimeEscapeError @with_pool pool begin + pool = _make_pool(true) + _lazy_checkpoint!(pool) + err = try mat = acquire!(pool, Float64, 10, 10) - _test_leak(mat) # opaque to compile-time checker; caught by runtime LV2 + _validate_pool_return(_test_leak(mat), pool) + nothing + catch e + e + finally + _lazy_rewind!(pool) end + @test err isa PoolRuntimeEscapeError # Raw Array from unsafe_acquire! should throw error when returned - @test_throws PoolRuntimeEscapeError @with_pool pool begin + pool = _make_pool(true) + _lazy_checkpoint!(pool) + err = try mat = unsafe_acquire!(pool, Float64, 10, 10) - _test_leak(mat) # opaque to compile-time checker; caught by runtime LV2 + _validate_pool_return(_test_leak(mat), pool) + nothing + catch e + e + finally + _lazy_rewind!(pool) end + @test err isa PoolRuntimeEscapeError # Safe returns should work fine - result = @with_pool pool begin + pool = _make_pool(true) + _lazy_checkpoint!(pool) + result = try mat = acquire!(pool, Float64, 10, 10) mat .= 1.0 sum(mat) # Safe: returning scalar + finally + _lazy_rewind!(pool) end @test result == 100.0 # Returning a copy is also safe - result = @with_pool pool begin + pool = _make_pool(true) + _lazy_checkpoint!(pool) + result = try mat = acquire!(pool, Float64, 3, 3) mat .= 2.0 collect(mat) # Safe: returning a copy + finally + _lazy_rewind!(pool) end @test result == fill(2.0, 3, 3) - - POOL_DEBUG[] = old_debug end # ============================================================================== @@ -321,45 +292,60 @@ _test_leak(x) = x # opaque to compile-time escape checker (only identity() is t rewind!(pool) end - @testset "POOL_DEBUG with BitArray" begin - old_debug = POOL_DEBUG[] - POOL_DEBUG[] = true - - # BitVector from pool should throw error when returned with debug on - @test_throws PoolRuntimeEscapeError @with_pool pool begin + @testset "Pool{1} escape detection with BitArray" begin + # BitVector from pool should throw error when returned + pool = _make_pool(true) + _lazy_checkpoint!(pool) + err = try bv = acquire!(pool, Bit, 100) - _test_leak(bv) # opaque to compile-time checker; caught by runtime LV2 + _validate_pool_return(_test_leak(bv), pool) + nothing + catch e + e + finally + _lazy_rewind!(pool) end + @test err isa PoolRuntimeEscapeError # BitMatrix from pool should throw error when returned - @test_throws PoolRuntimeEscapeError @with_pool pool begin + pool = _make_pool(true) + _lazy_checkpoint!(pool) + err = try ba = acquire!(pool, Bit, 10, 10) - _test_leak(ba) # opaque to compile-time checker; caught by runtime LV2 + _validate_pool_return(_test_leak(ba), pool) + nothing + catch e + e + finally + _lazy_rewind!(pool) end + @test err isa PoolRuntimeEscapeError # Safe returns should work fine - result = @with_pool pool begin + pool = _make_pool(true) + _lazy_checkpoint!(pool) + result = try bv = acquire!(pool, Bit, 100) bv .= true count(bv) # Safe: returning scalar + finally + _lazy_rewind!(pool) end @test result == 100 # Returning a copy is also safe - result = @with_pool pool begin + pool = _make_pool(true) + _lazy_checkpoint!(pool) + result = try bv = acquire!(pool, Bit, 5) bv .= true copy(bv) # Safe: returning a copy + finally + _lazy_rewind!(pool) end @test result == trues(5) - - POOL_DEBUG[] = old_debug end - # ============================================================================== - # POOL_DEBUG with function definition forms - # ============================================================================== - # ============================================================================== # _validate_pool_return — recursive container inspection (Tuple, NamedTuple, Pair) # ============================================================================== @@ -566,192 +552,98 @@ _test_leak(x) = x # opaque to compile-time escape checker (only identity() is t rewind!(pool) end - @testset "_validate_pool_return containers via @with_pool macro (LV2)" begin - old_safety = POOL_SAFETY_LV[] - set_safety_level!(2) # creates Pool{2} — safety level baked into type - - # Tuple containing pool array — caught at runtime - @test_throws PoolRuntimeEscapeError @with_pool pool begin + @testset "_validate_pool_return containers via Pool{1} (direct validation)" begin + # Tuple containing pool array — caught + pool = _make_pool(true) + _lazy_checkpoint!(pool) + err = try v = acquire!(pool, Float64, 10) - _test_leak((sum(v), v)) # opaque to compile-time checker; runtime LV2 catches v inside tuple + _validate_pool_return(_test_leak((sum(v), v)), pool) + nothing + catch e + e + finally + _lazy_rewind!(pool) end + @test err isa PoolRuntimeEscapeError - # NamedTuple containing pool array — caught at runtime - @test_throws PoolRuntimeEscapeError @with_pool pool begin + # NamedTuple containing pool array — caught + pool = _make_pool(true) + _lazy_checkpoint!(pool) + err = try v = acquire!(pool, Float64, 10) - _test_leak((data = v, n = 10)) # opaque to compile-time checker; runtime LV2 catches v inside NamedTuple + _validate_pool_return(_test_leak((data = v, n = 10)), pool) + nothing + catch e + e + finally + _lazy_rewind!(pool) end + @test err isa PoolRuntimeEscapeError # Safe containers pass - result = @with_pool pool begin + pool = _make_pool(true) + _lazy_checkpoint!(pool) + result = try v = acquire!(pool, Float64, 10) v .= 3.0 (sum(v), length(v)) + finally + _lazy_rewind!(pool) end @test result == (30.0, 10) - - set_safety_level!(old_safety) end # ============================================================================== - # Runtime LV2 escape detection through opaque function calls + # Pool{1} escape detection through opaque function calls (direct validation) # ============================================================================== - @testset "Runtime LV2 catches escapes through opaque function calls" begin - old_safety = POOL_SAFETY_LV[] - set_safety_level!(2) # creates Pool{2} — safety level baked into type - + @testset "Pool{1} catches escapes through direct validation" begin # Opaque function call bypasses compile-time PoolEscapeError, - # but runtime _validate_pool_return at LV2 still catches the escape. - @test_throws PoolRuntimeEscapeError @with_pool pool begin + # but direct _validate_pool_return on Pool{1} still catches the escape. + pool = _make_pool(true) + _lazy_checkpoint!(pool) + err = try v = acquire!(pool, Float64, 10) - _test_leak(v) # opaque to compile-time checker; runtime LV2 catches + _validate_pool_return(_test_leak(v), pool) + nothing + catch e + e + finally + _lazy_rewind!(pool) end + @test err isa PoolRuntimeEscapeError - # Multiple vars: opaque call still caught at runtime - @test_throws PoolRuntimeEscapeError @with_pool pool begin + # Multiple vars: validation still catches escape + pool = _make_pool(true) + _lazy_checkpoint!(pool) + err = try v = acquire!(pool, Float64, 10) w = acquire!(pool, Float64, 5) - _test_leak(v) + _validate_pool_return(_test_leak(v), pool) + nothing + catch e + e + finally + _lazy_rewind!(pool) end + @test err isa PoolRuntimeEscapeError # Safe return works fine - result = @with_pool pool begin + pool = _make_pool(true) + _lazy_checkpoint!(pool) + result = try v = acquire!(pool, Float64, 10) v .= 1.0 sum(v) # scalar — safe + finally + _lazy_rewind!(pool) end @test result == 10.0 - - set_safety_level!(old_safety) - end - - @testset "LV1 does not perform runtime escape check" begin - old_safety = POOL_SAFETY_LV[] - set_safety_level!(1) # creates Pool{1} — only structural invalidation - - # At LV1, opaque call bypasses compile-time and runtime doesn't check escapes - # (only structural invalidation), so the SubArray escapes silently. - result = @with_pool pool begin - v = acquire!(pool, Float64, 10) - _test_leak(v) - end - @test result isa SubArray # Escapes — no runtime check at LV1 - - set_safety_level!(old_safety) end # ============================================================================== - # POOL_DEBUG with function definition forms - # ============================================================================== - - @testset "POOL_DEBUG with @with_pool function definition" begin - old_debug = POOL_DEBUG[] - POOL_DEBUG[] = true - - # Unsafe: function returns pool-backed SubArray - @with_pool pool function _test_debug_func_unsafe(n) - v = acquire!(pool, Float64, n) - v .= 1.0 - _test_leak(v) # opaque to compile-time checker; caught by runtime LV2 - end - @test_throws PoolRuntimeEscapeError _test_debug_func_unsafe(10) - - # Safe: function returns scalar - @with_pool pool function _test_debug_func_safe(n) - v = acquire!(pool, Float64, n) - v .= 1.0 - sum(v) - end - @test _test_debug_func_safe(10) == 10.0 - - # Safe: function returns a copy - @with_pool pool function _test_debug_func_copy(n) - v = acquire!(pool, Float64, n) - v .= 2.0 - collect(v) - end - @test _test_debug_func_copy(5) == fill(2.0, 5) - - # Unsafe: N-D ReshapedArray from function - @with_pool pool function _test_debug_func_nd(m, n) - mat = acquire!(pool, Float64, m, n) - mat .= 1.0 - _test_leak(mat) # opaque to compile-time checker; caught by runtime LV2 - end - @test_throws PoolRuntimeEscapeError _test_debug_func_nd(3, 4) - - # Unsafe: BitVector from function - @with_pool pool function _test_debug_func_bit(n) - bv = acquire!(pool, Bit, n) - bv .= true - _test_leak(bv) # opaque to compile-time checker; caught by runtime LV2 - end - @test_throws PoolRuntimeEscapeError _test_debug_func_bit(100) - - POOL_DEBUG[] = old_debug - end - - @testset "POOL_DEBUG with @maybe_with_pool function definition" begin - old_debug = POOL_DEBUG[] - old_maybe = MAYBE_POOLING[] - POOL_DEBUG[] = true - MAYBE_POOLING[] = true - - # Unsafe: function returns pool-backed array - @maybe_with_pool pool function _test_maybe_debug_unsafe(n) - v = acquire!(pool, Float64, n) - v .= 1.0 - _test_leak(v) # opaque to compile-time checker; caught by runtime LV2 - end - @test_throws PoolRuntimeEscapeError _test_maybe_debug_unsafe(10) - - # Safe: function returns scalar - @maybe_with_pool pool function _test_maybe_debug_safe(n) - v = acquire!(pool, Float64, n) - v .= 1.0 - sum(v) - end - @test _test_maybe_debug_safe(10) == 10.0 - - # When pooling disabled, no validation needed (DisabledPool returns fresh arrays) - MAYBE_POOLING[] = false - @maybe_with_pool pool function _test_maybe_debug_disabled(n) - v = zeros!(pool, n) - _test_leak(v) # opaque to compile-time checker; disabled pool returns fresh arrays - end - result = _test_maybe_debug_disabled(5) - @test result == zeros(5) - - POOL_DEBUG[] = old_debug - MAYBE_POOLING[] = old_maybe - end - - @testset "POOL_DEBUG with @with_pool :cpu function definition" begin - old_debug = POOL_DEBUG[] - POOL_DEBUG[] = true - - # Unsafe: backend function returns pool-backed array - @with_pool :cpu pool function _test_backend_debug_unsafe(n) - v = acquire!(pool, Float64, n) - v .= 1.0 - _test_leak(v) # opaque to compile-time checker; caught by runtime LV2 - end - @test_throws PoolRuntimeEscapeError _test_backend_debug_unsafe(10) - - # Safe: returns scalar - @with_pool :cpu pool function _test_backend_debug_safe(n) - v = acquire!(pool, Float64, n) - v .= 1.0 - sum(v) - end - @test _test_backend_debug_safe(10) == 10.0 - - POOL_DEBUG[] = old_debug - end - - # ============================================================================== - # Coverage: PoolRuntimeEscapeError showerror with return_site (LV3) + # Coverage: PoolRuntimeEscapeError showerror with return_site # ============================================================================== @testset "PoolRuntimeEscapeError showerror with return_site" begin @@ -799,15 +691,12 @@ _test_leak(x) = x # opaque to compile-time escape checker (only identity() is t # Rational is not AbstractFloat, Integer, or Complex → hits generic fallback @test _poison_value(Rational{Int}) == zero(Rational{Int}) - # Exercise through actual pool rewind at LV≥2 with a non-fixed-slot type - old_lv = POOL_SAFETY_LV[] - set_safety_level!(2) - pool = AdaptiveArrayPool() - checkpoint!(pool) + # Exercise through actual pool rewind at S=1 with a non-fixed-slot type + pool = _make_pool(true) + _lazy_checkpoint!(pool) v = acquire!(pool, Rational{Int}, 5) v .= 1 // 3 - rewind!(pool) # triggers _poison_fill! → _poison_value(Rational{Int}) → zero(Rational) - set_safety_level!(old_lv) + _lazy_rewind!(pool) # triggers _poison_fill! → _poison_value(Rational{Int}) → zero(Rational) end # ============================================================================== @@ -822,4 +711,4 @@ _test_leak(x) = x # opaque to compile-time escape checker (only identity() is t @test occursin("42", loc) end -end # POOL_DEBUG and Safety Validation +end # Safety Validation diff --git a/test/test_macro_expansion.jl b/test/test_macro_expansion.jl index 539bb9bb..3e1b7f78 100644 --- a/test/test_macro_expansion.jl +++ b/test/test_macro_expansion.jl @@ -78,8 +78,8 @@ @test occursin("Int64", expr_str) end - # Test POOL_DEBUG validation in block mode - @testset "POOL_DEBUG validation in expansion" begin + # Test _runtime_check validation in block mode + @testset "Runtime check validation in expansion" begin expr = @macroexpand @with_pool pool begin v = acquire!(pool, Float64, 10) sum(v) @@ -87,7 +87,7 @@ expr_str = string(expr) - # POOL_DEBUG is inlined as RefValue, but _validate_pool_return should be present + # _runtime_check gates _validate_pool_return (dead-code eliminated when S=0) @test occursin("_validate_pool_return", expr_str) end diff --git a/test/test_safety.jl b/test/test_safety.jl index 7a7faa41..d3465f39 100644 --- a/test/test_safety.jl +++ b/test/test_safety.jl @@ -1,29 +1,24 @@ -import AdaptiveArrayPools: _invalidate_released_slots!, PoolRuntimeEscapeError +import AdaptiveArrayPools: _invalidate_released_slots!, PoolRuntimeEscapeError, _make_pool, _validate_pool_return # Opaque identity — defeats compile-time escape analysis without @skip_check_vars _test_leak(x) = x -@testset "POOL_SAFETY_LV Guard-Level Invalidation" begin +@testset "RUNTIME_CHECK Guard-Level Invalidation" begin # ============================================================================== # Default values # ============================================================================== @testset "Default configuration" begin - # DEFAULT_SAFETY_LV depends on LocalPreferences.toml (0 when absent) - @test POOL_SAFETY_LV[] == DEFAULT_SAFETY_LV - @test STATIC_POOL_CHECKS == (DEFAULT_SAFETY_LV > 0) + @test RUNTIME_CHECK isa Int end # ============================================================================== - # Level 1: acquire! SubArray invalidation + # S=1: acquire! SubArray invalidation # ============================================================================== @testset "acquire! SubArray invalidated on rewind" begin - old_safety = POOL_SAFETY_LV[] - set_safety_level!(1) - - pool = AdaptiveArrayPool() + pool = _make_pool(true) checkpoint!(pool) v = acquire!(pool, Float64, 10) v .= 42.0 # write to confirm it's valid before rewind @@ -34,15 +29,10 @@ _test_leak(x) = x # Accessing stale SubArray should throw BoundsError @test_throws BoundsError v[1] - - set_safety_level!(old_safety) end @testset "acquire! N-D ReshapedArray invalidated on rewind" begin - old_safety = POOL_SAFETY_LV[] - set_safety_level!(1) - - pool = AdaptiveArrayPool() + pool = _make_pool(true) checkpoint!(pool) mat = acquire!(pool, Float64, 5, 5) mat .= 1.0 @@ -51,21 +41,16 @@ _test_leak(x) = x # Parent chain: ReshapedArray -> SubArray -> Vector (now length 0) @test length(parent(parent(mat))) == 0 @test_throws BoundsError mat[1, 1] - - set_safety_level!(old_safety) end # ============================================================================== - # Level 1: unsafe_acquire! Array wrapper invalidation (Julia 1.11+ only) + # S=1: unsafe_acquire! Array wrapper invalidation (Julia 1.11+ only) # On Julia 1.10, Array is a C struct — setfield!(:size) is not available. # ============================================================================== @static if VERSION >= v"1.11-" @testset "unsafe_acquire! Array wrapper invalidated on rewind" begin - old_safety = POOL_SAFETY_LV[] - set_safety_level!(1) - - pool = AdaptiveArrayPool() + pool = _make_pool(true) checkpoint!(pool) arr = unsafe_acquire!(pool, Float64, 10) arr .= 99.0 @@ -75,15 +60,10 @@ _test_leak(x) = x # Wrapper size set to (0,) via setfield! @test size(arr) == (0,) @test_throws BoundsError arr[1] - - set_safety_level!(old_safety) end @testset "unsafe_acquire! N-D Array wrapper invalidated on rewind" begin - old_safety = POOL_SAFETY_LV[] - set_safety_level!(1) - - pool = AdaptiveArrayPool() + pool = _make_pool(true) checkpoint!(pool) mat = unsafe_acquire!(pool, Float64, 4, 3) mat .= 1.0 @@ -92,20 +72,15 @@ _test_leak(x) = x @test size(mat) == (0, 0) @test_throws BoundsError mat[1, 1] - - set_safety_level!(old_safety) end end # ============================================================================== - # Level 1: BitArray invalidation + # S=1: BitArray invalidation # ============================================================================== @testset "acquire! BitVector invalidated on rewind" begin - old_safety = POOL_SAFETY_LV[] - set_safety_level!(1) - - pool = AdaptiveArrayPool() + pool = _make_pool(true) checkpoint!(pool) bv = acquire!(pool, Bit, 100) bv .= true @@ -115,15 +90,10 @@ _test_leak(x) = x @test length(pool.bits.vectors[1]) == 0 # Accessing stale BitVector - len was set to 0 via setfield! @test length(bv) == 0 - - set_safety_level!(old_safety) end @testset "acquire! BitMatrix invalidated on rewind" begin - old_safety = POOL_SAFETY_LV[] - set_safety_level!(1) - - pool = AdaptiveArrayPool() + pool = _make_pool(true) checkpoint!(pool) ba = acquire!(pool, Bit, 8, 8) ba .= true @@ -133,19 +103,14 @@ _test_leak(x) = x # BitArray dims set to (0, 0), len set to 0 @test size(ba) == (0, 0) @test length(ba) == 0 - - set_safety_level!(old_safety) end # ============================================================================== - # Level 0: No invalidation + # S=0: No invalidation # ============================================================================== - @testset "POOL_SAFETY_LV=0 bypasses invalidation" begin - old_safety = POOL_SAFETY_LV[] - set_safety_level!(0) - - pool = AdaptiveArrayPool() + @testset "S=0 bypasses invalidation" begin + pool = _make_pool(false) checkpoint!(pool) v = acquire!(pool, Float64, 10) v .= 7.0 @@ -155,8 +120,6 @@ _test_leak(x) = x @test length(parent(v)) >= 10 # Stale access works (this is the unsafe behavior we're protecting against) @test v[1] == 7.0 - - set_safety_level!(old_safety) end # ============================================================================== @@ -164,10 +127,7 @@ _test_leak(x) = x # ============================================================================== @testset "Re-acquire after invalidation restores vectors" begin - old_safety = POOL_SAFETY_LV[] - set_safety_level!(1) - - pool = AdaptiveArrayPool() + pool = _make_pool(true) # First cycle: populate pool checkpoint!(pool) @@ -187,16 +147,11 @@ _test_leak(x) = x # Same backing vector object (capacity preserved through resize round-trip) @test parent(v2) === pool.float64.vectors[1] rewind!(pool) - - set_safety_level!(old_safety) end @static if VERSION >= v"1.11-" @testset "Re-acquire unsafe_acquire! after invalidation" begin - old_safety = POOL_SAFETY_LV[] - set_safety_level!(1) - - pool = AdaptiveArrayPool() + pool = _make_pool(true) # First cycle checkpoint!(pool) @@ -212,8 +167,6 @@ _test_leak(x) = x arr2 .= 4.0 @test arr2[1] == 4.0 rewind!(pool) - - set_safety_level!(old_safety) end end @@ -222,10 +175,7 @@ _test_leak(x) = x # ============================================================================== @testset "Nested checkpoint/rewind: inner invalidated, outer valid" begin - old_safety = POOL_SAFETY_LV[] - set_safety_level!(1) - - pool = AdaptiveArrayPool() + pool = _make_pool(true) checkpoint!(pool) v_outer = acquire!(pool, Float64, 10) v_outer .= 1.0 @@ -247,8 +197,6 @@ _test_leak(x) = x # Now outer is also invalidated @test length(parent(v_outer)) == 0 - - set_safety_level!(old_safety) end # ============================================================================== @@ -256,23 +204,18 @@ _test_leak(x) = x # ============================================================================== @testset "reset! invalidates all active slots" begin - old_safety = POOL_SAFETY_LV[] - set_safety_level!(1) - - pool = AdaptiveArrayPool() + pool = _make_pool(true) checkpoint!(pool) v1 = acquire!(pool, Float64, 10) v2 = acquire!(pool, Float64, 20) v1 .= 1.0 v2 .= 2.0 - reset!(pool.float64) + reset!(pool.float64, 1) # S=1 to trigger invalidation @test pool.float64.n_active == 0 @test length(pool.float64.vectors[1]) == 0 @test length(pool.float64.vectors[2]) == 0 - - set_safety_level!(old_safety) end # ============================================================================== @@ -280,10 +223,7 @@ _test_leak(x) = x # ============================================================================== @testset "Fallback type invalidation" begin - old_safety = POOL_SAFETY_LV[] - set_safety_level!(1) - - pool = AdaptiveArrayPool() + pool = _make_pool(true) checkpoint!(pool) v = acquire!(pool, UInt8, 50) v .= 0xff @@ -294,45 +234,6 @@ _test_leak(x) = x tp = pool.others[UInt8] @test length(tp.vectors[1]) == 0 @test length(parent(v)) == 0 - - set_safety_level!(old_safety) - end - - # ============================================================================== - # POOL_DEBUG backward compatibility - # ============================================================================== - - @testset "POOL_DEBUG backward compat with POOL_SAFETY_LV" begin - old_debug = POOL_DEBUG[] - old_safety = POOL_SAFETY_LV[] - - # POOL_DEBUG=true still triggers escape detection (regardless of POOL_SAFETY_LV) - POOL_DEBUG[] = true - set_safety_level!(0) - @test_throws PoolRuntimeEscapeError @with_pool pool begin - v = acquire!(pool, Float64, 10) - _test_leak(v) # bypasses compile-time check; caught by runtime LV2 - end - - # POOL_SAFETY_LV=2 also triggers escape detection (without POOL_DEBUG) - POOL_DEBUG[] = false - set_safety_level!(2) - @test_throws PoolRuntimeEscapeError @with_pool pool begin - v = acquire!(pool, Float64, 10) - _test_leak(v) # bypasses compile-time check; caught by runtime LV2 - end - - # Neither flag -> no escape detection - POOL_DEBUG[] = false - set_safety_level!(1) - result = @with_pool pool begin - v = acquire!(pool, Float64, 10) - _test_leak(v) # bypasses compile-time check; runtime LV<2 won't catch - end - @test result isa SubArray - - POOL_DEBUG[] = old_debug - set_safety_level!(old_safety) end # ============================================================================== @@ -340,10 +241,7 @@ _test_leak(x) = x # ============================================================================== @testset "Multiple types invalidated together" begin - old_safety = POOL_SAFETY_LV[] - set_safety_level!(1) - - pool = AdaptiveArrayPool() + pool = _make_pool(true) checkpoint!(pool) vf = acquire!(pool, Float64, 10) vi = acquire!(pool, Int64, 20) @@ -356,8 +254,6 @@ _test_leak(x) = x @test length(parent(vf)) == 0 @test length(parent(vi)) == 0 @test length(vb) == 0 - - set_safety_level!(old_safety) end # ============================================================================== @@ -365,38 +261,25 @@ _test_leak(x) = x # ============================================================================== @testset "@with_pool invalidates on scope exit" begin - old_safety = POOL_SAFETY_LV[] - set_safety_level!(1) - - pool_ref = Ref{AdaptiveArrayPool}() - stale_ref = Ref{Any}() - - result = @with_pool pool begin - pool_ref[] = pool - v = acquire!(pool, Float64, 10) - v .= 5.0 - stale_ref[] = v - sum(v) # Safe scalar return - end + pool = _make_pool(true) + checkpoint!(pool) + v = acquire!(pool, Float64, 10) + v .= 5.0 + result = sum(v) # Safe scalar return + rewind!(pool) @test result == 50.0 - # After @with_pool exits, the pool's vectors should be invalidated - v = stale_ref[] + # After rewind, the pool's vectors should be invalidated @test length(parent(v)) == 0 @test_throws BoundsError v[1] - - set_safety_level!(old_safety) end # ============================================================================== - # Level 2: Poisoning (NaN/sentinel fill before structural invalidation) + # S=1: Poisoning (NaN/sentinel fill before structural invalidation) # ============================================================================== - @testset "Level 2: Float64 poisoned with NaN on rewind" begin - old_safety = POOL_SAFETY_LV[] - set_safety_level!(2) - - pool = AdaptiveArrayPool() + @testset "Float64 poisoned with NaN on rewind" begin + pool = _make_pool(true) checkpoint!(pool) v = acquire!(pool, Float64, 10) v .= 42.0 @@ -408,15 +291,10 @@ _test_leak(x) = x v2 = acquire!(pool, Float64, 10) @test all(isnan, v2) rewind!(pool) - - set_safety_level!(old_safety) end - @testset "Level 2: Int64 poisoned with typemax on rewind" begin - old_safety = POOL_SAFETY_LV[] - set_safety_level!(2) - - pool = AdaptiveArrayPool() + @testset "Int64 poisoned with typemax on rewind" begin + pool = _make_pool(true) checkpoint!(pool) v = acquire!(pool, Int64, 10) v .= 42 @@ -426,15 +304,10 @@ _test_leak(x) = x v2 = acquire!(pool, Int64, 10) @test all(==(typemax(Int64)), v2) rewind!(pool) - - set_safety_level!(old_safety) end - @testset "Level 2: ComplexF64 poisoned with NaN+NaN*im on rewind" begin - old_safety = POOL_SAFETY_LV[] - set_safety_level!(2) - - pool = AdaptiveArrayPool() + @testset "ComplexF64 poisoned with NaN+NaN*im on rewind" begin + pool = _make_pool(true) checkpoint!(pool) v = acquire!(pool, ComplexF64, 8) v .= 1.0 + 2.0im @@ -444,29 +317,6 @@ _test_leak(x) = x v2 = acquire!(pool, ComplexF64, 8) @test all(z -> isnan(real(z)) && isnan(imag(z)), v2) rewind!(pool) - - set_safety_level!(old_safety) - end - - @testset "Level 1 does NOT poison" begin - old_safety = POOL_SAFETY_LV[] - set_safety_level!(1) - - pool = AdaptiveArrayPool() - checkpoint!(pool) - v = acquire!(pool, Float64, 10) - v .= 42.0 - rewind!(pool) - - # At level 1, only resize (no poison). Re-acquire restores length, - # data is whatever was in memory — should still be 42.0 (not NaN). - checkpoint!(pool) - v2 = acquire!(pool, Float64, 10) - @test !any(isnan, v2) - @test v2[1] == 42.0 - rewind!(pool) - - set_safety_level!(old_safety) end -end # POOL_SAFETY_LV Guard-Level Invalidation +end # RUNTIME_CHECK Guard-Level Invalidation diff --git a/test/test_state.jl b/test/test_state.jl index 8c9fa11a..d4124b64 100644 --- a/test/test_state.jl +++ b/test/test_state.jl @@ -437,7 +437,7 @@ import AdaptiveArrayPools: _typed_lazy_checkpoint!, _typed_lazy_rewind!, _tracke @test length(tp._checkpoint_n_active) > 1 # Reset TypedPool directly - result = reset!(tp) + result = reset!(tp, 0) @test result === tp @test tp.n_active == 0 @test tp._checkpoint_n_active == [0] @@ -674,7 +674,7 @@ import AdaptiveArrayPools: _typed_lazy_checkpoint!, _typed_lazy_rewind!, _tracke @test tp.n_active == 1 v2 = acquire!(pool, Float64, 200) @test tp.n_active == 2 - _rewind_typed_pool!(tp, 1) + _rewind_typed_pool!(tp, 1, 0) @test tp.n_active == 0 # Nested checkpoint/rewind on TypedPool @@ -690,13 +690,13 @@ import AdaptiveArrayPools: _typed_lazy_checkpoint!, _typed_lazy_rewind!, _tracke v3 = acquire!(pool, Float64, 30) @test tp.n_active == 3 - _rewind_typed_pool!(tp, 3) + _rewind_typed_pool!(tp, 3, 0) @test tp.n_active == 2 - _rewind_typed_pool!(tp, 2) + _rewind_typed_pool!(tp, 2, 0) @test tp.n_active == 1 - _rewind_typed_pool!(tp, 1) + _rewind_typed_pool!(tp, 1, 0) @test tp.n_active == 0 # Verify type-specific checkpoint delegates to TypedPool diff --git a/test/test_task_local_pool.jl b/test/test_task_local_pool.jl index a3b75739..d4ef765b 100644 --- a/test/test_task_local_pool.jl +++ b/test/test_task_local_pool.jl @@ -208,81 +208,24 @@ MAYBE_POOLING[] = old_state end - @testset "set_safety_level! preserves cached arrays" begin - old_lv = POOL_SAFETY_LV[] - try - # Start at level 0, populate the pool - set_safety_level!(0) - pool0 = get_task_local_pool() - @with_pool pool begin - acquire!(pool, Float64, 10) - acquire!(pool, Float32, 5) - acquire!(pool, Int64, 3) - end - # Snapshot references from old pool - old_f64 = pool0.float64 - old_f32 = pool0.float32 - old_i64 = pool0.int64 - old_bits = pool0.bits - old_others = pool0.others - n_f64 = length(old_f64.vectors) - n_f32 = length(old_f32.vectors) - n_i64 = length(old_i64.vectors) - - @test n_f64 >= 1 - @test n_f32 >= 1 - @test n_i64 >= 1 - - # Switch to level 2 — cache should survive - pool2 = set_safety_level!(2) - @test pool2 isa AdaptiveArrayPool{2} - @test pool2 !== pool0 # different object (new S) - - # TypedPool references are identical (not just equal) - @test pool2.float64 === old_f64 - @test pool2.float32 === old_f32 - @test pool2.int64 === old_i64 - @test pool2.bits === old_bits - @test pool2.others === old_others - - # Slot counts unchanged - @test length(pool2.float64.vectors) == n_f64 - @test length(pool2.float32.vectors) == n_f32 - @test length(pool2.int64.vectors) == n_i64 - - # Switch back to 0 — still preserved - pool0b = set_safety_level!(0) - @test pool0b isa AdaptiveArrayPool{0} - @test pool0b.float64 === old_f64 - - # Borrow log reset on non-level-3 - @test pool2._borrow_log === nothing - # Level 3 gets fresh borrow log - pool3 = set_safety_level!(3) - @test pool3._borrow_log isa IdDict - @test pool3.float64 === old_f64 # cache still same - finally - set_safety_level!(old_lv) - end - end - - @testset "set_safety_level! with no existing pool" begin - # Remove existing pool to test the nothing path - old_lv = POOL_SAFETY_LV[] - tls = task_local_storage() - old = get(tls, AdaptiveArrayPools._POOL_KEY, nothing) - try - delete!(tls, AdaptiveArrayPools._POOL_KEY) - pool = set_safety_level!(1) - @test pool isa AdaptiveArrayPool{1} - # Should have created a fresh empty pool - @test length(pool.float64.vectors) == 0 - finally - if old !== nothing - tls[AdaptiveArrayPools._POOL_KEY] = old - end - POOL_SAFETY_LV[] = old_lv - end + @testset "_make_pool creates correct pool types" begin + pool0 = AdaptiveArrayPools._make_pool(false) + @test pool0 isa AdaptiveArrayPool{0} + + pool1 = AdaptiveArrayPools._make_pool(true) + @test pool1 isa AdaptiveArrayPool{1} + @test pool1._borrow_log === nothing # lazily initialized on first borrow + + # _make_pool(Bool, old) preserves cached arrays + pool0_with_data = AdaptiveArrayPools._make_pool(false) + AdaptiveArrayPools._lazy_checkpoint!(pool0_with_data) + acquire!(pool0_with_data, Float64, 10) + AdaptiveArrayPools._lazy_rewind!(pool0_with_data) + + old_f64 = pool0_with_data.float64 + pool1_from_old = AdaptiveArrayPools._make_pool(true, pool0_with_data) + @test pool1_from_old isa AdaptiveArrayPool{1} + @test pool1_from_old.float64 === old_f64 # same TypedPool reference end @testset "Pool growth warning at 512 arrays" begin diff --git a/test/test_zero_allocation.jl b/test/test_zero_allocation.jl index bf28dc76..fe4db45b 100644 --- a/test/test_zero_allocation.jl +++ b/test/test_zero_allocation.jl @@ -387,10 +387,9 @@ const _ZERO_ALLOC_THRESHOLD = @static VERSION >= v"1.12-" ? 0 : 16 # Pattern 7: @inline @with_pool function form (regression test) # # When @inline is applied to a @with_pool function, the compiler inlines - # everything into the caller — including the _dispatch_pool_scope closure. + # everything into the caller — including the let-block pool binding. # This can defeat LLVM's escape analysis, causing SubArray metadata to be - # heap-allocated instead of stack-allocated. The @noinline closure fix in - # _wrap_with_dispatch preserves the function barrier. + # heap-allocated instead of stack-allocated. # ============================================================================== # Non-inlined baseline: acquire! + similar! + in-place ops