Cache watched rg file inventories

[jif-oai]

Why

Repeated rg --files calls rescan the same repository metadata. On a 34 GB monorepo that costs about 3.5 seconds per call, and concurrent agent turns make the contention worse.

This is the second PR in the transparent exec-server rg acceleration stack. It adds the cache mechanism behind the packaged shim, but deliberately does not expose it to executor children yet; the next PR owns that sandbox boundary.

What

• add a process-scoped cache manager that asks bundled stock rg to build the canonical --files inventory
• register the recursive filesystem watcher before the initial build and publish only a generation that saw no mutations
• invalidate readiness before every rebuild and stop serving the cache on watcher errors
• coalesce mutation storms through a one-slot signal while an atomic generation counter preserves correctness
• let the shim serve only the exact rg --files form, only at the canonical Git worktree root, and only without RIPGREP_CONFIG_PATH
• delegate every unsupported, unavailable, or invalid cache case to bundled stock rg

The cache is ephemeral, private to one process, and contains only stock rg output. No index format or search semantics are introduced here.

Safety boundaries

• the worktree root marker must exactly match the canonical current directory
• readers verify the ready-generation token both before and after opening the inventory
• the inventory is renamed into place before the ready marker
• access-only filesystem events are ignored; all mutation events increment the generation
• cache misses and failures preserve existing behavior by executing stock rg

Benchmarks

Measured on Apple Silicon / macOS 26.5.2 with ripgrep 15.1.0 against /Users/jif/code/openai at 5ee72035d863ff9f79a10735be60a1a07066cc9f (34 GB, 962,820 visible paths). The candidate was release-built from 944c9cf7947a7e6f17b1e3cb850688ecab9e9bc1.

The initial watched inventory took 3.756 s to build and occupied 73,143,023 bytes. Stock and cached results had the same line count, byte count, and unordered SHA-256 digest. The benchmark used two warmups, randomized pair order, bootstrap confidence intervals for the paired median speedup, and exact two-sided sign tests.

scenario	stock rg median	cached median	paired speedup (95% CI)	n	sign-test p
serial	3,492.8 ms	23.9 ms	145.1x (141.7–152.9x)	12 pairs	0.00049
2 concurrent calls	5,997.2 ms	36.0 ms	164.7x (159.2–176.2x)	6 pairs	0.03125
4 concurrent calls	11,683.9 ms	47.1 ms	246.2x (238.3–253.4x)	6 pairs	0.03125

The concurrent figures are whole-batch wall times. A separate live watcher check created and removed a file after publication; both refreshed inventories matched stock rg, with 37.6 ms and 38.8 ms observed refresh latency respectively. That check is a correctness probe, not a statistically powered latency claim.

Checks

• just test -p codex-rg --features manager (6 passed)
• just bazel-lock-check from an exact detached worktree at this commit
• cargo metadata --locked --no-deps --format-version 1 from that exact worktree
• just fmt

Stack

• base: Route packaged rg through exec-server runtime #31932
• next: exec-server lifecycle and sandbox-policy integration