Out-of-memory persistent storage: the tree learns to live in a KV store - #8
Open
plaidfinch wants to merge 904 commits into
Open
Out-of-memory persistent storage: the tree learns to live in a KV store#8plaidfinch wants to merge 904 commits into
plaidfinch wants to merge 904 commits into
Conversation
…elled to_rank Ranked's equality is version identity while a bare Rank can only speak rank classes, and one rank class holds many versions: an == between the two types cannot satisfy PartialEq's cross-type transitivity contract (rank-equal distinct views r1, r2 would give r1 == k and k == r2 with r1 != r2). The comparison surface is homogeneous on both types, the type docs carry the explicit spelling (a.to_rank() == k, a.to_rank().cmp(&k)), and the law and known-value pins denominate the rank question through that spelling. In-memory comparison surface only: no encoding changes, board rows untouched.
… name The exemption tiles the coverage roster by exact op name; the row is the homogeneous Ranked comparison family plus the Ranked / Rank From conversions.
The (Merged, Merged) arm of span_all's balanced hull fold first fires at four total inputs, and every committed correctness surface stopped at arity 3 (the law's triple, the causally tests' pairs, the meter pins' inequality-only arity 9): a lo-leg misread of a merged group's endpoint survived before's entire 707-test all-features suite, demonstrated by mutation (a_lo.meet_view(b_hi) for b_lo), and was caught in the workspace only by rumors' tree differential (range_and_freeze_match_the_naive_filter), whose shrunk fringe-branch seed is committed here. rumors' fringe bounds memo calls span_all over up to 256 leaf children, so the arm is a production path. The law now folds a fourth clause: a.span_all([b, c, a, b]) must equal the triple hull — the five inputs are repeats, not lattice derivatives (b & c / b | c items are absorbed by b and c and leave a misread invisible; verified green under the mutation before this shape was chosen), so the counter's two weight-1 groups hull(a, b) and hull(c, a) each carry information the other lacks in both directions. Adequacy: the mutation above reads red through the new clause (version_triple_laws) and green without it; both runs performed. Span::owned's coverage note re-states what the laws now actually pin: every combine arm, the merged-merged one through the arity-5 clause.
span_shares_the_crossing_folds claimed its limb-meter undercut (110 < 68 + 74) pins the fused hull's shared accumulator, arguing a two-accumulator spelling 'would read exactly the composed sum'. It would not: accumulator folds record zero limb ops (Base's Magnitude impl meters nothing; suanpan traffic is the touch meter's), so the limb undercut is entirely the decode side's per-value width records. A constructed two-accumulator pseudo-fusion — cursors advanced once, each crossing folded into two accumulators — read byte-identically to the true kernel on every committed span pin (limbs 110, scan identity 203 + 101 == 304, span_all 1817 < 2178) while sharing nothing, and passed the emit byte differentials: a wrong artifact every criterion blessed. The limb leg's comment now states what it pins (the single decode at arithmetic width; witness: an unfused hull, which reads the composed 142 exactly), and a touch leg pins the fold sharing: the fake reads the composed sum there exactly (26 == 13 + 13, verified red) where the true kernel undercuts (21 < 26, verified green).
…ctly The emit test module's doc claims the fused hull sweep rides every oracle comparison; the organic-histories proptest checked join and meet against the oracle without it. The hull now rides there as well, making the claim true in the strengthening direction. Version::span's complexity doc claimed 'half the scan and accumulator traffic' of composing & and |; the round's own pins refute the totals reading (scan 203 vs 304, touches 21 vs 26 — the outputs are emitted either way). The prose now states the exact saving: the composition's second pair decode and second set of crossing folds, with the meter suite's two faces (scan identity exact, touch undercut) cited for what each pins.
…sarial review's three fixes
… own account A new gate leg (just surface-totality) parses nightly rustdoc JSON for before --all-features — ground truth with no source-file list to forget — and holds every public function-like item (free fns, inherent methods, public-trait-declared methods) to exactly one disposition: a METHOD_SURFACE roster row or a named, dated exception. Trait-impl methods stay FAMILY_SURFACE's review-governed domain. Tooling: rustdoc-types =0.59.0 (the official schema crate), pinned exact as the format pin; the checker refuses any document whose format_version differs, naming both numbers, and the recipe comment documents the nightly-bump procedure. public-api was evaluated first and rejected for cause: its latest release pins rustdoc-types ^0.57 (format 57), behind the workspace nightly's format 59, and its rendered-token items carry no structured impl context to resolve the roster's Type::fn naming against. The checker is a detached workspace (the fuzzfit idiom) reading the roster through before/meter — one enumeration, no second copy to drift. Exceptions are self-pruning (dead entries fail), discipline-checked (undated or thin rulings fail), and prefix-scoped with :: so a module exception can never leak to a sibling; two liveness anchors keep an empty or wrong-tree walk from reading green. First-run census: 189 public function-like items = 90 rostered (every METHOD_SURFACE row live, both directions) + 99 under three module-scope exceptions (meter:: 65, oracle:: 32, surface:: 2), zero item exceptions. Tripwires demonstrated: removing the Party::is_seed roster row reads red naming the row; a format_version 60 document is refused by name; ten committed unit tests drive every finding category.
The declared-cells binding of record already stands: the board smoke suite's board_runs_to_completion derives cells-per-family from one live board sweep and holds every registry row's declared Coverage::Board number to it, in the gate via test-all (landed with the registry door, af316ee). Adequacy re-demonstrated under a reversible mutation: dense declared 50 -> 49 reads red naming the family and both maps; restored, green, diff empty. What had drifted was the prose beside the binding: the cells field's class breakdown still counted the operation table before the span_all fold row and the overlap fold row landed (49/70/3 against the enforced 50/71/4), and OpGroup::Fold's doc named three fold rows of five. Both corrected toward the code; the fold-population reach is stated with the four row names its bundles actually feed (version_join_all, version_meet_all, version_span_all, party_join_all — measured from the rendered board, party_join_all_overlap riding its own bundle).
… tier Audit of Version::join_all/meet_all's shared balanced fold (the Group operand-form arms): (input, input) first fires at arity 2, the merged-input drain at arity 3, the in-counter merged-merged combine at arity 4 — and (input, merged) is unreachable by the counter's weight discipline, kept total for commutativity rather than asserted. Under a reversible value-corrupting mutation of the merged-merged arm (drop the newer group), the committed differentials all read red — join_all/meet_all_equals_the_sequential_fold, meet_all_matches_oracle, and the deterministic shade-population pin (both feed orders place the carrier in a dropped group, so both drop directions lose it) — but version_triple_laws read GREEN: no law reached arity 4, the span_all review's genre one tier down, and the law collection is what the fuzz target drives over hostile-but-canonical decodes. Landed fold_all_arity_five_matches_the_pair_folds in VERSION_TRIPLE: join_all/meet_all over [a, b, c, a, b] — repeated raw inputs, never lattice derivatives, so the two weight-1 groups (a-b and c-a) each carry an input the other lacks — compared against the sequential pair operators, which cannot share a broken arm. Adequacy: the same mutant now fails version_triple_laws naming exactly this law; restored, green, diff empty. The two shrunk fold-mutation witnesses the differentials produced (4-fork worlds, the smallest merged-merged populations) ride along as committed seeds, replaying the arm through both sequential-fold differentials on every run.
The new law's and the module-exceptions table's first paragraphs rendered past the one-liner budget; the tails move below a blank line as body prose, content unchanged.
Two owner-directed public additions to causally::Span, closing the gap that forced foreign span consumers into per-read trusted-door construction: reborrow() lends a shorter-lived Span over the same endpoints (no clone, no validation — lo <= hi rides through from the source), and into_owned() settles the borrows to a Span<'static>, following the Ranked::into_owned inherent-method precedent. Registry discipline: both are clauses of the span_is_the_pair_hull law (reborrow identity and settled-endpoint preservation, byte-equal via Version's byte-equality Eq, with into_owned exercised in both borrow states), both carry doctests and pinned Complexity lines, and both are rostered — surface rows, complexity claims, and board not-applicable entries (accessors, not metered walks: no board rows).
…an accessor The trait's floor()/ceiling()/dominance_of() triple becomes a single span() -> causally::Span<'_>: the node's version bounds as one causal span, riding before's new reborrow door. The load-time contract that lived on dominance_of — ordering priced once at construction, Span::new for validated load with Crossed as the storage-corruption error, Span::ordered as the trusted door, a violated ordering meaning unspecified verdicts and silently wrong reconciliation — moves onto the span accessor: it is the trait's whole version-bounds obligation now. The tree grows the matching inherent (untyped::Node::span / typed::Node::span): a branch reborrows its stored bounds memo, a leaf answers the coincident span through the trusted door; the in-tree inherent ceiling/floor/dominance_of and their memo mechanics are untouched, so the pinned classifier and memo-fold readings hold byte-identical (fused classifier scan 7804; memo-fold x4/x3 margins). All four impls collapse (Local by inherent delegation, Failing/Charged/ Materializing by wrapper delegation). Classifiers ask the span dominance_of directly; single-endpoint consumers take meet()/join() off a span bound to a local where the endpoint must outlive the read (wire encode's leaf version, the conformance corpus ceiling fold via a Vec of borrowing spans) and read through the temporary elsewhere (the containment checks). No wire or verdict change anywhere: the span's join IS the ceiling the same sites read before.
Owner ruling (2026-07-31): the trust-the-caller constructor door takes the standard Rust naming convention — new_unchecked, the unchecked counterpart of Span::new — keeping its debug_assert door semantics and contract docs exactly. Swept every reference workspace-wide: the causally module and Span docs, the hull laws' call sites, the surface roster row, the complexity claim, the board coverage entry, the causally unit tests (names and expected panic message), rumors' leaf span construction and the Backend trait's trusted-door sentence, and before-fuelscape's exemption row. Also completes the fuelscape coverage tiling for the two new Span doors (reborrow, into_owned), which the fuelscape parity test demands beside the surface roster rows.
…code The span joins the codec surface: encode/encode_to emit the meet's canonical bytes then the join's (Clock's byte-aligned no-length-prefix framing; the causally module doc carries the wire form and the deliberate absence of a coincident-span short form), and decode is ONE forward pass — parse the meet, then parse the join through the new admission walk (skyline::admit), which validates dominance in the same walk that parses the stream. A span parsed from the wire is valid by construction; crossed and concurrent pairs reject as Decode::NotCanonical, the composite genre documented at the variant (the Ranked rank-mismatch precedent: the canonical spelling of no value). The admission walk drops the standalone validator's height accumulator outright rather than fusing it: an accepted walk holds height_hi >= height_lo >= 0 per elementary interval (dominance gives the left inequality, lo's own canonicality the right), so negative heights are always also non-dominating and share the rejection genre. The walk maintains only the pair sweep's running difference, with the sweep's own seed, step, and fold order, so its accumulator traffic is causal_cmp's exactly; truncation and collapsible-sibling topology ride the checked cursor's own reads, and a refuted dominance drops lo's cursor while the strict parse completes alone (structural genres win on multiply-defective input). Borsh gains the raw-framed composite Serialize and an owned Deserialize through the same fused kernel over the wire-side cursor.
…d-decode meter legs Laws: span_codec_roundtrip joins VERSION_PAIR — encode is the meet's encoding then the join's (stated), decode∘encode is the identity exactly, on the pair's hull and the coincident span, with re-encode byte equality. Verdict identity: the fused wire decode agrees with the composed form (decode, decode, Span::new) on accept and reject — exhaustively over every ordered pair of small-scope normal-form events (accept and reject both proven live at scale), and over arbitrary oracle pairs by proptest. Rejection witnesses committed per genre: strictly crossed, concurrent (both orders), non-canonical component on each side of the seam (the collapsible pair that denotes the empty version, so only canonicality can reject it as a join — the check the fusion must not lose), truncation at every byte boundary (inside the meet, at the seam, inside the join), trailing byte, and a set padding bit in each component's final byte. The composite's prefix-freedom is pinned directly on the composite. Borsh: composes field-by-field in a larger stream, genres cross the boundary intact, arbitrary hulls round-trip. Meter pins (relational, no constants to rot): scan — fused reads exactly decode(lo) + cmp(lo,hi) (130 == 42 + 88 on the fixture) and undercuts composed by one whole second-component parse; limb — fused sits between its own pieces and composed (66 in [54, 80]), the gap above the floor being the topology-minimality zero-equality the strict parse keeps; touch — fused folds exactly decode(lo) + cmp (14 == 3+11) while parse-then-validate reads the composed 17: the second validation-height accumulator is deleted outright, and the touch leg is the one a pseudo-fusion cannot fake.
span_encode joins the board as the composite emission's row (the codec emission genre: heap at the materialized composite, denominated by the span's own packed size) and span_decode as the fused parse-and-validate row, its floors the first component's parse plus the pair comparison's (both endpoints materialize, every stored payload decodes once, the whole composite examined, the pair's nonzero deltas folded) — the second component's standalone validation accumulator is what the fusion deletes, so no floor demands it. The rejection surface gets rows per the error-path precedent: span_decode_truncated, span_decode_trailing, and span_decode_crossed — the genre this decode mints, the reversed composite, with its own scan-floor derivation (the fused decode parses the whole composite before pronouncing the pair rejection). The registry's declared bundle reach gains the five new rows on every version-bearing family (50 -> 55, 71 -> 76, benign 75 -> 80; id-pair and the fold-only bundles untouched), and the Coverage doc's bundle figures are re-stated to the live board (they had lagged one round). Worst-map pins for the ten new entries transcribed from the live release fold at both scales, with the movement annotation; every pre-existing entry verified unmoved (the render diff at both scales is exactly the 140 new green cells, the new scan-floor legend line, and the cell-count footer). Surface roster rows, complexity claims (Linear, celled to the new rows), and fuelscape exemptions cover the three new public methods.
Seven over-long first paragraphs split into summary plus body; no content change.
…plan Round #148 (doc-only). The re-accreted execution history goes to git history; the document reads top to bottom as what remains, why, and how completion is judged, written against the tree at 4624bcc: - Section 14/17.2 re-derived to the real remaining tail (the Span wire-format review, the fuelscape causally-kernel round, the step! retirement with its ratchet obligation, the variadic law suites, the Bytes representation round, the tick-seam cache probe, the closeout obligations, the remote survey/soak, the final adversarial review lens deck with carried items, the wall-clock legs and before/after table, the prose cycle) plus go-criteria for main. - Section 12 consolidated: all 51 prior entries preserved (four merged into round-level records, none deleted), execution chronicle condensed to decisions and landed shapes, plus a post-#91 landed-shape digest and this restructure's dated entry. - Sections 2/3/13/17.3 re-derived to the code: the registry as the family roster of record, the empty red-triage buffer and the declared-model roster replacing the stale red accounting and every hand-maintained cell/family tally, board sharding and the worst-case map, the current judge roster and rider constant. - Stale values corrected toward the tree throughout (suanpan as the accumulator's home, the materialization row names, memwatch defaults, the flat fuzz heap cap's actual state).
… walk that works The dissolved blanket — no guest kernel exports the causally ops — may no longer excuse an operation that does real walk work. Six kernels and their registry-derived panels land with their measurements: - ff_version_span / ff_version_span_all: the fused pair hull and the n-ary hull fold (Version::span, Version::span_all), the span_all kernel following the join_all/meet_all register-range arity convention with the receiver fed first. The slice row carries both committed fold shapes — the staggered populations (the join endpoint's swell) and the meet shade (the meet endpoint's diagonal, carrier first) — since the hull fold drives both lattice directions in one pass. - ff_span_place / ff_span_dominance: the fused three-stream placement walk at full nine-way resolution and its early-exit dominance coarsening, each measured against a span composed in unmeasured preparation as two sampled operands' pair hull, with a third sampled version as the probe; a new overlay mapping crosses the committed pair shapes with a same-scale probe. - ff_span_encode / ff_span_decode: the composite emission and the strict parse whose fused admission walk proves the pair ordered in the same pass (the clock codec rows' compose-encode-decode staging precedent). The causally census: every exemption that remains carries its own reason with no blanket citation. Range::bounded / placement_of / contains stay exempt as the same fused placement co-walk under branch-only verdict hooks, now priced at the span_place panel (two bounds) and the version_cmp panel (the one-bound degenerate form), both identities meter-pinned; Span::new stays exempt because its validation is literally one Version PartialOrd call around an O(1) construction — a panel would re-measure version_cmp under another name; the O(1) constructors, accessors, and borrow-settling conversions state their shapes; Span::encode_to is priced at the new span_encode panel. Dissolved exemptions become covers entries, so the tiling pin holds the new totality (52 panels tile the roster with the exemption table). Existing readings are untouched: a fixed-seed 40-sample/64-byte survey at the parent guest and at this one renders all 46 pre-existing panels byte-identical, the fuzzfit enforcement suite is green against the unchanged pinned bands (no re-pin), and the amp board's measured code is byte-untouched (no change under crates/before/src).
The admission walk now returns the dominance verdict instead of pronouncing the pair rejection itself, and both callers (the byte-slice decode and the borsh leg) check their tail obligations — the composite's zero padding, the final byte's dead bits — before minting NotCanonical from a refuted verdict. Before this, a crossed pair with a set padding bit or a spurious trailing zero byte rejected NotCanonical where decoding the components rejects TrailingBits, contradicting both the composed form and the documented precedence (structural genres win on multiply-defective input). New goldens pin the precedence per genre: crossed+padding and crossed+trailing are TrailingBits, crossed+truncated is Truncated, a whole negative-height join is NotCanonical (the subsumption seam), and a negative-height join that is also truncated is Truncated — the one deliberate divergence from component-wise decoding, since the fused walk carries no height accumulator. An off-corpus verdict-identity sweep (300-level spines, 1024-leaf fans, payloads past the word window, 63/64-bit zigzag edges) and a span-level single-bit mutation proptest extend the identity's reach beyond the exhaustive corpus.
The serialization census (owner scope-add): borsh covers all six codec-bearing types, serde covered only Party/Version/Clock. Fill the gap following serde_impls.rs's own precedent exactly — each type serializes as its canonical byte encoding via serialize_bytes and deserializes back through its strict decode, so the serialized form is the one wire form and a deserialized value is canonical (and, for Span, valid) by construction. Span rides the same one-pass fused validate-while-parsing kernel as the raw and borsh paths, through Span::decode over the format-delivered bytes. The suite mirrors the borsh legs: round-trips through serde_json, postcard, and ciborium's typed byte string (major type 2); the postcard payload pinned byte-identical to encode(); strict rejection of trailing, rank-mismatch, and crossed payloads through both deserialization paths; and composition inside a larger serde value. The serde/borsh family claim now prices the rank/ranked/span codec cells alongside the original six.
…rose The borsh genre suite gains the crossed-plus-padding witness (the structural genre crosses the boundary ahead of the pair verdict, as the raw decode orders them). The worst-map movement annotation's date matches the landing commit. The streaming Backend's serialized-bounds prose now names Span::decode as the one-call load door for bounds stored as canonical span bytes, keeping Span::new for bounds held as two decoded versions.
…af delta Measured across 1/7/13-leaf joins the gap is 0/12/24 — exactly two limb ops per stepped second-component leaf and none for the first, which the admission walk never asks (its opening payload is absolute, no delta to test).
…p! retirement
Every operation the 11 step-scaling proptests guard (party/tests.rs: 6,
version/tests.rs: 5 — the brief's ~13 was a miscount at this base) has
replacement coverage in BOTH surviving instruments: a pinned fuzzfit fuel
band (fuel = actual wasm instructions, opt-out-proof) and amplification
board rows whose scan-bits column is live (exponent fitted across the
cell's two scales, liveness floor committed per cell).
step proptest public op fuzzfit band board row(s)
split_is_linear Party::fork/forks ff_party_fork(+s) party_fork
sum_is_linear Party::join ff_party_join party_join, party_join_all
sum_split_is_at_most_linear Clock::sync ff_clock_sync clock_sync
is_disjoint_is_linear Party::is_disjoint ff_party_is_disjoint party_disjoint
covers_is_linear Party::covers ff_party_covers party_covers
diff_is_linear Party::without ff_party_without party_without
leq_is_linear Version partial_cmp ff_version_cmp version_cmp, version_eq, version_concurrent
tick_is_linear Version::tick ff_version_tick version_tick, version_ticks
grow_bushy_is_linear tick's grow branch ff_version_tick version_tick_adv_party (Version::new() vs deep family party = the grow arm, every party family)
merge_is_linear Version | Version ff_version_join(_all) version_join(_assign,_all)
meet_is_linear Version & Version ff_version_meet(_all) version_meet(_assign,_all)
No op is uncovered; no stop-and-report gap. Flagged check resolved: no
recv/sync law leg rides the step machinery — workspace-wide grep for
steps_of/assert_linear_scaling/testing::metrics finds only lib.rs,
testing/{metrics,complexity}.rs, party/tests.rs, version/tests.rs. The
clock recv/sync resource record lives in tests/meter.rs (its own meters)
and ff_clock_recv/ff_clock_sync bands; it stays untouched.
… board scan column Retirement-ratchet demonstrations, one per walk genre the step suite covered. Each mutant was applied as a reversible Edit string swap, measured through the amplification board (release, limb-meter+scan-meter), and restored by the reverse Edit (git diff against the pre-mutation commit verified empty after each). Baselines: scale 1 all green (1846 cells); scale 0.25 (the gate determinism recipe's committed scale) has 73 pre-calibration reds, so 0.25 readings are new-red diffs against the same-scale baseline. 1. Party pruned structural walk: IdReader::read (idbits.rs) re-scans the stream remainder per node read. Instrument: board, scale 1 (all-green baseline). Reading: 104 RED cells; party_disjoint/id-pair scan[e 1.00, 8.0/B] -> scan[e 2.00, 95996.0/B], mech[exponent+constant] <- scan exponent, scan constant; same on party_covers, party_join, version_tick_adv_party (13 families), et al. 2. Version overlay walk: the comparison sweep (sweep.rs fn sweep) re-scans the left stream per elementary interval. Instrument: board, scale 0.25. Reading: 41 new RED cells vs same-scale baseline; version_cmp/dense scan[e 1.00, 8.0/B] -> scan[e 2.00, 31989.4/B], mech[exponent+constant]; version_cmp +24 families, version_concurrent +17. 3. Query fold (a LeafWalk driver client): max_depth (query.rs) re-scans the whole stream per visited leaf. Instrument: board, scale 0.25. Reading: 166 new RED cells vs same-scale baseline; version_rank/dense scan[e 1.00, 18.7/B] -> scan[e 2.00, 32021.3/B]; version_rank +24, version_distance/version_lag +23 each, ranked_* rows (embedded rank) +23 to +25. The fuel leg's adequacy is separately committed: the fuzzfit enforce sentry's standing self-check requires a deliberately quadratic guest burner to read ABOVE a linear band on every run, and fuel meters actual executed instructions, so no walk can opt out by omission — the failure mode that motivated this retirement.
Owner-ruled retirement (2026-07-31): the cfg(test) step counter is
entirely subsumed by instruments that cannot be fooled by a forgotten
macro call — fuzzfit fuel bands meter actual executed wasm instructions
over the whole public surface, and the amplification board's scan-bits
column prices every packed-stream traversal with fitted exponents and
per-cell liveness floors. The opt-in hole was real: a walk without the
macro read zero steps and the ratio test passed vacuously. The
retirement ratchet was satisfied first (phase 1: per-op coverage table;
phase 2: three constructed re-scan quadratic mutants, one per walk
genre, each reading RED through the board scan column).
Deleted:
- the step! macro (both cfg twins) and its pub(crate) re-export (lib.rs)
- testing/metrics.rs (the thread-local counter)
- testing/complexity.rs (steps_of, assert_linear_scaling, MIN_SCALE)
- the 11 step-scaling proptests: party/tests.rs split/sum/is_disjoint/
covers/diff/sum_split; version/tests.rs leq/tick/grow_bushy/merge/meet
- all 27 production step!() call sites (idbits, party/ops/{index,diff,
split}, skyline/{fill,grow,query,text,sweep,admit,walk}) and the two
in text/tests.rs's schoolbook adequacy kernel
- covers_stress_pair (its only consumer was the deleted covers test);
skip_stress_pair survives (the deep indexed-disjointness differential)
- every prose reference to traversal-step counting, re-denominated
toward the surviving instruments; design-doc decision records exempt
The deep indexed-disjointness differential keeps its own DEEP_SCALE
constant where it previously borrowed MIN_SCALE. Proptest seed files
untouched (hard rule). No public API change; the board is untouched
production-surface-wise (the counter was cfg(test)-only).
The implementation moves out of a monolithic lib.rs into four modules, re-exported at the crate root so every existing use path keeps compiling unchanged: accumulator (the digit buffer, every operation, the collapsing sign fold, the zero-run ledger, and the touch seam), limbs (the 64-bit limb view over dashu's stored words), magnitude (the caller-facing width seam), and touch_meter (the feature-gated process-global counter, at its same public path). The crate docs stay on lib.rs, which now carries only declarations and re-exports. The accumulator's tests move to its sibling tests file per convention, and their proptest seed moves with the module so the committed counterexample keeps reproducing. The claims roster's source-of-record list, site paths, and witness paths follow the items they pin; the rendered lines, table cells, and every pinned touch count are unchanged.
…er boundary
Four new signature groups in the law collection, each quantifying over
list arity — the variable every fixed-arity group caps:
- VERSION_LIST: join_all/meet_all against their sequential pair-operator
folds (empty/unary/binary edges are the same equations at k = 0/1/2),
and order-independence over every rotation plus the reversal.
- VERSION_AND_LIST: span_all as the family hull (endpoints definitionally
meet_all/join_all over {receiver} ∪ items, accessors exact, every input
within) and rotation-independence with each element taking a turn as
receiver.
- PARTY_AND_LIST: join_all acceptance iff family pairwise disjointness
(accepted folds equal the sequential pair joins), fork reunification at
every width (with a proper-subregion value comparison against the
sequential fold), and per-input lossless best-effort rejection at every
width.
- CLOCK_AND_LIST: the clock counterparts — acceptance iff disjoint
parties with both components sequential-fold-equal, and ticked fork
families reuniting to the original region with the join of every line.
The drivers sweep arity across the balanced counter's structural
boundaries, derived from the fold's own shape and documented at
arb_fold_arity: short-circuits (0, 1), first leaf combine (2), first
drain (3), first merged-merged carry (4), first drain merged-merged (6),
plus two octaves (8/9, 15/16/17). Lists index small pools (the empty
version always present), so repeats — the raw inputs a misread merged
operand visibly loses — and aliased parties arise at every arity.
Wired to the arbitrary-normal-form drivers (new pool-family strategies)
and the organic-population driver (pool = the trace population).
… both arity octaves The law target's framing gains three list scripts after the six decoded chunks — [arity: u8][pool indices], versions then parties then clocks, arity folded into the fold-boundary band (0..=17) and indices into small pools of the decoded values, missing bytes reading as zero — so decoded hostile-but-canonical values drive the variadic law groups at every counter boundary, with repeats and aliases arising from the pools. The committed corpus updates with the framing (regenerated by the fuzz_seeds example from the extended derivation): laws_family carries first-octave scripts (an arity-5 version list crossing the merged-merged carry and the drain, arity-4 party/clock lists with aliased repeats); laws_wide_gamma carries second-octave scripts (17/16/15). The corpus gate now parses the scripts per the framing, holds every seed byte in-band as written, and pins both octave tails alongside the wide-gamma one.
The observer test prose is re-denominated in terms of the owned inherent next and the Stream face; the Send-bounds test's second leg names StreamExt::next explicitly so it exercises the face its doc claims (the inherent method was shadowing it into a duplicate of the first leg). Two new pins: a no-change commit (no-op redact, empty batch) produces no observer tick while a live observer still ticks for real commits, and the walk differential's internal-entry choice is documented as a deliberate decision at the check site.
…itnesses The V1 protocol gate moves to the session entry points, before the preamble: an unsupported protocol/backend pairing now declines with nothing written and nothing read, making ProtocolUnsupported's before-any-wire-traffic contract literally true; the in-arm gates remain as total re-checks. Bootstrap persists the absorbed tree and identity before certifying the epilogue, so a provider can never hold a completion certificate for an absorption that never landed. Both party guards publish through send_if_modified returning false: party motion without tree change wakes no observer, as the bookmark reclaim already promised. Store::commit's implementor contract gains the two clauses a persistent backend must not 'fix': the recorded clock may lag growth-ward only (crash cost is a leaked region, never a duplicated one), and the backend may only serialize the alias it is handed. The conformance suite's deferred-law list picks up the PERSISTS/commit coherence coupling and the durable-identity shrink law. New witnesses, each mutation-verified: a storage-backed peer declines V1 one-sidedly on both the gossip and bootstrap paths (borrowing the conformance suite's Materializing as the crate's one storage-owning backend), and a gossip session cancelled with a received retiree donation parked behind the commit lock recovers the identity through the absorb guard rather than stranding it.
transport_failures_are_exact_and_fail_fast now replays a shrunk case (fail_left, operation Accept, after 0, chunk 1, unbuffered) whose injected fault surfaces through an unexpected error path. Verified failing at 94788a6b in a scratch worktree before any of this branch's storage work existed: the defect is pre-existing, discovered here only because a fresh proptest run explored the case. Committed red per the instruments-before-cures rule; the cure is a triage item, not part of the storage-foundations change that follows.
…tody The bottom half of the persistence layer, everything below the tree backend: - store::kv — the public Kv contract (the Link of storage): named byte tables, atomic serializable write transactions and snapshot reads over sync closures taking &mut dyn cursor views, a prefix-consistency crash floor, and a no-op-default durability barrier. Closures may re-run; dropped writes are committed-or-not, never partial. - store::Memory — the reference store, with the two instruments the crash tests need: a committed-state history making every prefix a reopenable crash survivor, and injectable faults (abort; commit-then-error, the acknowledgment ambiguity made deterministic). - conformance::kv — the public factory-based suite for caller stores (visibility, read-your-writes, abort atomicity, cursor order, serialization under concurrency, dropped-write all-or-nothing), with three lying stores as negative controls proving each probe's teeth by name. - store::schema — the four tables and their borsh records: the canonical root carrying the durable identity as inert clock bytes (a record layer that can serialize an identity but structurally cannot tick or join it), node records with eager memos and fat child edges, held-table presence rows keyed by (node, pin) so releases are idempotent, and the block-reserving ID allocator that names nodes before their transactions run. - store::refcount — custody: strong counts equal durable edges plus the root edge; live handles are held rows; reclamation drains a persistent queue in bounded steps (cascades are queue entries, never transaction growth); recovery unconditionally sweeps the held table and never touches the canonical root. The crash-point sweep proptest reopens every committed prefix of generated custody workloads, recovers, vacuums, and audits: strong counts recomputed from edges, no dangling references, storage equal to the canonical root's reachable closure, identity equal to the last committed write-or-clear. Mutation-verified: a skipped child decrement and a skipped recovery sweep each fail the audit. Every gate tier is green except the pre-existing defect pinned by the previous commit, excluded by name from this commit's test-all run.
KvBackend implements Backend + Store over the schema and custody layers: one decoded record per resident node behind a weak dedup funnel, height-typed views whose offsets walk compressed spans with zero I/O and whose hashes derive at mint time from the record alone, and pending bodies that stage construction in memory until a branch install or root flip links them — collapsing every singleton-extension chain to exactly one record per span, the shape the in-memory tree holds. Store::commit gains the network (the canonical record is what a reopened store reconstructs a peer from); Store::record and Store::barrier are the party-shrink write and the durability barrier the donation paths and the fork section now run before identity-bearing bytes cross the wire. Peer::open resumes a stored replica: recovery sweep, canonical record, identity clock, root entry handle; an empty store and a retired one refuse distinctly. Two custody lessons are pinned in code: a mid-span reification's registration must survive until the transaction that links it (the guard now travels with the ID), and incrementing an absent record panics as the accounting bug it is rather than installing a dangling edge. Hash preimages take spans in path order while storage is shallowest-last, exactly as the in-memory node hashes. Behavioral suite: sends/reads/redactions through every public face, heterogeneous Local-KV gossip convergence over the existing wire in both directions, reopen-and-resume with post-restart convergence against a witness, Empty/Retired open refusals, and a quiescence audit pinning storage to exactly the canonical root's closure.
…census The store differentials gain a third replica over KvBackend<Memory>: the same generated action sequences now pin the persistent backend's act, get, range, and (same-store) join against the in-memory engines, and the session-conformance census runs the KV backend through the full controlled-divergence reconciliation with a Measure oracle — node_bytes bounds every measured handle pointwise, payload excluded on both sides per its contract. The deferred-law ledger is re-denominated: the shrink, custody, and sequencing laws now live with the backend's own battery suite, where the store's committed history makes each falsifiable at every crash prefix; the one structural obligation Rust cannot test from outside (the PERSISTS-commit coherence coupling) stays a stated contract.
…the shrink law Every committed transaction prefix of a real workload reopens to a consistent replica (audit invariants, walkable tree, a state the live run actually published); injected store faults abort commits cleanly (published tree unchanged, no observer wakeup, StorageError surfaced, clean audit after recovery); the ambiguous-commit window resolves forward; sends dropped at every poll depth are full-or-nothing with the party never lost. The durable-identity shrink law lands with transaction-ordering teeth, mutation-verified: a bootstrap-serving donation must reach the identity record in a transaction strictly before the session's closing flip (deleting the record write turns the test red), and a retirement's clear obeys the same once-cleared-never-resurrected monotonicity across every crash prefix. Memory gains clear_faults so batteries end their fault windows before auditing honest behavior. The future-size guardrail gains the KV entries: the storage calls on the session path box their frames (the donation record, the install persist, the mutation seams), and the KV budget rides at the same order of magnitude with the honest handle delta documented — the failure mode the guardrail exists to catch still lands in the tens of KiB.
…urface The crate docs gain the persistence section: bring a Kv, validate it with conformance::kv, seed_in/backend/open, the crash-consistency floor vs the store's durability policy, and the deployment obligations (one process, snapshot custody, vacuum). The outgrows-its-smallest-peer non-goal is re-denominated for stored replicas: a store lifts the RAM ceiling, never the full-replica model. KvBackend documents space custody with a runnable vacuum example. The feature matrix checks the conformance surface compiled alone, and sync-budget.md records how the storage boundary actually landed (Kv public, Backend internal) as a dated amendment.
The rebase onto radix-fan main replayed sixteen commits (the proxy terminal cure was already upstream and dropped); this commit carries the seams the replay could not express per-commit: - imbl returns as an optional dependency riding the conformance and test-internals features, exactly the features that expose Memory (the reference store whose crash corpus is imbl's structural sharing); production builds compile no imbl at all. The upstream-defect pin test keeps asserting the defect exists and flips when imbl fixes it. - The diff deny-list becomes mechanical: clippy.toml disallows the diff entry points by def-path (adequacy-verified against the pin test's own call), and the new tools/denylist gate tier bans the token in every cfg combination. The == residual is stated in clippy.toml rather than guarded by a rule that cannot see types: both probed forms failed their witness tests, and a guard that catches nothing is decoration. - The session-overlap instruments speak the async surface: sends and redactions await through the house idioms, and the deterministic witness gains a mechanism-derived calibration floor (>= 8 alternation rounds; measured 10 post-flip) so a harness that loses its poll granularity reads as red, not as vacuously green. - The internal witness re-expresses over the Local oracle faces; the routed-link mesh awaits its sends; the proxy failure seeds are the union of both lines' witnesses; the swap-publish comments re-derive without naming the walk they retired.
… clone Every read surface — Snapshot::get, the Messages stream, both observers — now yields (Key, Arc<Version>, Arc<T>): the leaf stores its version behind one shared handle (both backends: the in-memory leaf and the persistent backend's decoded record), and Leaf::version is the accessor the yields ride. Serialization, bounds, hash preimages, and comparisons read through the deref; wire bytes, census, and window pricing are untouched. PROVISIONAL: a before-crate refactor in flight may make Version Bytes-backed and (near-)free to clone, which would obsolete this interning. The seam is deliberately thin — leaf construction and the yield sites — so backing it out is a localized reverse edit. The read-path regression this cures (measured on ox-east-1, criterion medians, pset-quiet, in_memory): branch-side pre-tuning numbers are pinned in the criterion records of this campaign's measurement round; the movement table lands with the re-measure in the follow-up commit.
Store gains an associated Walk type — the concrete stream range returns — so the read surfaces (Messages, both observers' passes) hold each backend's walk unboxed. Local's Walk is a synchronous owned leaf walk behind an always-ready Stream face: no box, no vtable, no executor round-trip per item. Storage-owning backends keep the boxed generic walk (ranged is its one-line range body). The tree of this commit is byte-identical to the state the full gate verified before the A1/A2 split (git write-tree f9d7d66), so the gate's verdict carries over exactly. Measured (ox-east-1, x86_64, pset-run -n 8, criterion medians; main = 8f4eecb and branch pre-tuning = 1393c96 in sibling build dirs, tuned = this commit + its parent, back-to-back under the same regime): bench pre-tuning/main tuned/main get 100/10k/1M 2.7 / 2.3 / 2.1 2.6 / 2.1 / 2.0 iter 100/10k/1M 4.7 / 4.2 / 2.2 4.3 / 3.0 / 1.7 range_delta d<<n 1.1 - 1.4 1.1 - 1.4 range_delta d=n cells 4.6 / 4.5 4.1 / 3.4 (µs-scale at n<=10k) observer_replay 1.9 - 2.1 1.7 - 1.8 causal_replay 1.4 - 2.0 1.2 - 1.8 batch_insert 1.00 1.03 - 1.05 The interning + static dispatch bought 8-29% on walks and replays and ~5% on point gets; the residual gap to the sync-read era is the async surface itself (a Local get spends more in future machinery than the whole borrowing lookup did), not version cloning — this bench mints single-party versions, where the clone was already cheap. Writes pay 1.03-1.05x for the per-leaf Arc allocation (the A1 provisionality note on the parent commit covers backing that out).
The poll-atomicity comments at both publish sites now name the operative indivisible pair — persist → publish, not build → publish — and state why a drop parked inside the persist is benign (store-ahead roots never reach the wire). The Kv contract's ownership clause sharpens from single-process to its real requirement, at most one live backend per store's tables with in-process handles included, and seed_in, open, and Bootstrap::backend each echo the hazard at their own surface. One assert literal loses a run of spaces from a missing continuation backslash, and CausalMessages::next spells its owned yield type the way its unordered sibling does.
Durable causal subscriptions: an arbitrary number of named, independently-consumed, rank-ordered backlogs stored beside the tree, replacing the causal observer's unbounded in-memory staging for persistent peers. Rank gains a canonical order-preserving wire encoding in before (filed to land off before-hardening); rows are pin-table references with epoch-major visibility and deletion-as-cursor delivery. Design only: the roadmap, verification program, and open questions ride inside the document.
…ry join The reconciliation the replay onto before-hardening main could not express per-commit; each seam resolves toward both sides' intent. - The stored branch record's two bound Versions become one causally::Span in its canonical composite encoding: decoding a record is now itself the validating load door (Span's borsh deserialize runs the fused one-pass parse), so corrupt bytes and crossed pairs refuse the node under the record layer's corruption policy, and no stored pair ever reaches the trusted door unvalidated. KvNode answers the collapsed Node::span obligation by reborrowing the stored span; a leaf answers the coincident span through the trusted door, exactly as the in-memory leaf does. Parent assembly folds the children's borrowing spans (meet of meets, join of joins, by-ref balanced folds), mirroring the in-memory bounds memo. - Store::join and the generic join tower thread the changed flag with the sync traversal's exact mechanics (drops detected by the filter's own shed observation on our side, any survivor a gain on theirs), so every commit path — batch, gossip write-back, the KV backend's override — answers "did the tree change?" from the flag with no root hash read. The conformance join differential pins the flag's biconditional against the hash oracle per orientation and holds the three backends' flags in agreement; the point-lookup and range legs compare whole spans, strengthening the old ceiling-only equality. - The tree's flag pins ride the Local oracles (join_now/react_now/act return the flag); the in-memory fringe fold reads through the interned leaf versions; the record fixtures build the empty coincident span.
The rebase onto before-hardening IS the convergence the doc provided for, so the doc now reads against the tree it sits in: - Section A becomes the requirements-of-record over the landed codec — order preservation pinned four ways, embedding safety with the adversarial review's refuted-draft provenance, strict canonical decode, byte-literal goldens, borsh and serde transports, the Ranked composite precedent, provenance-linear size — each cited by law name; the mechanism stays in the rank rustdoc of record, not here. - Ruling 1 carries a done-by-convergence status (including where the landed mechanism superseded the ruling-time sketch: in-house decoder, inverted-polarity Elias delta); roadmap B0 is marked done the same way; the cherry-pick-confluence risk is gone with the interregnum. - The appendix's decision record is untouched (the A1 clause is another round's charter).
…count bump Owner ruling 2026-07-30 (task #16): before's #120 made Version Bytes-backed, so a plain clone shares the packed encoding at O(1) -- the exact property the Arc<Version> handles bought, pinned in before by join_all_equal_operands_is_clone_cheap's flatness leg. No cross-node dedup existed (handles were minted per leaf), nothing compared version pointers, and the equality fast paths ride the Bytes sharing a plain clone preserves, so the interning's whole remaining economy was a 40-byte handle copy per yield. Every read surface -- Tree::get/get_now, Snapshot::get, the Messages stream, both observers, testing::collect/collect_range -- yields owned (Key, Version, Arc<T>) again; the Leaf::version accessor hands out &Version; both backends store the version inline in the leaf body. Wire bytes, census, and window pricing are untouched.
The future-size note carried a dated movement annotation (old -> new constants and a comparison against a retired representation): provenance that belongs in git history. The comment now states the structural fact the budget rests on — the bounds-span memo's two 40-byte Version handles dominate the branch variant — which stays true however the constants move next.
The landed convention had every decode door in the store layer panic on undecodable bytes, rationalized as only-writer trust. That rationale is circular: bit rot, torn writes a store's integrity layer misses, and an operator's misdirected script all falsify the only-writer premise with no programmer erring — so corruption is environmental, and the crate's job is to report it, not to decide for the deployment by dying. The redesign splits the two storage-failure genres a deployment handles oppositely. KvError<E> is now the persistent backend's error type: KvError::Store wraps the Kv implementation's own failure (retry against healthy storage), KvError::Corrupt carries a Corruption naming the table and key of a row the store returned that no backend ever wrote (retrying cannot help; content is recoverable from any replica by gossip). Every decode door refuses through it: node records (the span field's validating borsh parse included, so crossed bounds refuse exactly as garbled bytes do), the canonical-root record, the identity clock, the ID-allocator row and its overflow, node-ID and held row keys, and leaf payloads. A shape door at the walk-entry seams (child fetches and the root load) additionally refuses records whose stored span disagrees with the height of the edge naming them, which is what keeps the height-zero leaf accessors' branch arm structurally unreachable rather than corruption-reachable. A refusal must apply nothing: corruption can surface mid-transaction, after mutations were issued (reclamation decodes between its deletes). The Kv contract fixes closure errors to the store's own type, so the new store::checked module runs every custody transaction through views that carry KvError as their error channel and buffer mutations until the closure succeeds — a refusing closure leaves the store byte-for-byte untouched, proven by a differential proptest holding the checked view observationally identical to direct writes. Peer::open surfaces corruption as its own OpenError::Corrupt arm, distinct from Empty and Retired. The custody detector panics (the absent-record fetch, adjust_strong's accounting asserts) are deliberately unchanged: they fire on violations of the documented exclusive-ownership requirement, a genre flagged for a separate ruling. Witnesses: a truncated record and a byte-constructed crossed-span record each refuse as the observable error naming their row (plus the layout-premise control for the crossed construction), and a rotted canonical record refuses Peer::open with the corrupt row named.
Linearity is a hard constraint in every deployment: once a version crosses the wire, a restart that re-mints its coordinate is causal nonsense, whatever the store's durability policy. The pre-transmit barrier in the session driver already forced Kv::sync before every gossip transmission; this change gives the backend the watermark pair that makes the discipline precise and cheap, states the resulting guarantee in the crate docs, and pins it with a fault battery. KvBackend now tracks committed (identity-bearing commits acknowledged: root flips and identity records) against durable (the committed count a completed Kv::sync has covered). Store::barrier compares them: a send window containing no new commit flushes nothing — lazy stores keep their commit-latency win — while any wire send carrying versions newer than the durable watermark forces exactly one sync first. The count is snapshotted before the flush so commits acknowledged mid-flush are never marked durable by it. The crate docs' durability split becomes: acknowledged-and-escaped implies durable, for every store policy — a commit a crash takes is one whose versions never crossed the wire — and the identity-atomicity sentence now holds unconditionally (identity is durable before anything it stamped reaches a peer). Memory grows the matching instrumentation face: a durability ledger (sync_count, synced_prefix — reopen_at(synced_prefix()) is the worst crash a write-behind store's policy permits) and a one-shot sync fault. The battery pins the watermark discipline (one flush per commit-then-send window, none for local churn or commit-free windows), the escape invariant (the worst legal crash still holds everything a session shipped, with the flush strictly before the session's closing transactions), and unconstructibility (a failing sync blocks the session before anything escapes; the counterparty sees a dead session and no unflushed content).
…dule The Kv contract's re-execution clause obliges every transaction closure to route all effects through its transaction argument and tolerate re-execution from scratch — but until now nothing held the crate's own custody closures to it: no committed instrument would catch a closure leaking an effect into captured state, because the reference store runs each closure exactly once. Memory gains the retrying face: the optimistic-conflict schedule, execute against a discarded view, then execute again, deterministic and entirely inside the contract (the public kv conformance suite passes over it — hostile but legal, pinned). The backend batteries now run over the retrying store: the Store differentials against the synchronous engines, the crash-prefix and fault sweeps, the cancellation battery, the shrink laws, and the session census — so any backend closure with an unrouted effect diverges under assertions that already exist. Adequacy witnesses pin the schedule itself: a deliberately leaking closure observably runs twice while the committed state reflects one application (with a plain-store control), and a failing closure aborts once, unretried. The schedule lives as a Memory instrumentation face rather than a wrapper Kv type: a distinct store type would instantiate the whole height-indexed protocol tower a second time, the cost the conformance knobs exist to avoid, and Memory is already the crate's home for instrumented storage (history, faults, the durability ledger). The design appendix's post-merge task record now states the landed shape. Also carried here: the failing-sync battery from the wire-barrier commit now hangs up its link when the failing side aborts, so the counterparty's half-run session resolves as a dead session instead of parking forever on a silent peer (caught on that battery's first full run).
Two owner rationales join the docs in teaching register. The V1/storage gate now says why the pairing is refused rather than allowed to run: V1's full-level alternation assumes a resident tree, and over a storage backend it would fault nodes into memory with no window bounding them — in the worst case materializing the whole store to send a single message, the cost V2's fixed-memory streaming exists to bound. The rationale lives at the ProtocolUnsupported variant (the canonical surface), the error table's row, and both session gates. The deployment obligations gain the backup guidance: you basically never restore a store from a backup copy — a state revert rolls identity back with content, resurrecting donated identity and re-minting coordinates other replicas may hold. Gossip is the backup that preserves the causal guarantees: replicate to a second peer and recover a lost store by bootstrapping fresh and synchronizing. The obligations paragraph's opener no longer carries a hand-maintained tally.
Each over-long first paragraph splits at its claim: the summary keeps the invariant's headline, the elaboration moves below the fold.
…acked Tree The campaign added its deep changed-flag witnesses against the storeless Tree; on this branch every Tree carries a backend and join is awaited. The three sites take the in-memory Local backend and the join_now door, exactly as the flag's original witness already does.
The trunk retired the chatroom demo and its iroh stack; this branch's lock now straddles only the rand_core 0.6 / getrandom 0.2 pair that before's rand 0.8 dev pin holds against imbl and proptest's current rand_core line — restored here with the surviving holdouts named. rand itself resolves to one version on this branch, so its skip goes.
Same genre as the changed-flag depth tests: a campaign-added test called the storeless join; on this branch join is awaited and the synchronous door is join_now.
…tdoc gate The docs-internal leg documents private items with warnings denied, which the storage chain's module docs had never been built under: one out-of-scope Corruption link, three redundant explicit targets, and three write links ambiguous against the std macro.
… tree::backend::local Proptest resolves seed files by module path; the seeds stay live at the module's new home, and the seed-liveness pin agrees.
plaidfinch
force-pushed
the
persistent-storage
branch
from
July 31, 2026 22:40
9eaaef0 to
ffc321c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this is
Out-of-memory persistent storage for rumors: the same copy-on-write structural sharing the in-memory tree has always had, now generic over a storage backend, with a transactional-KV persistence layer beneath it. A peer can hold a message set larger than RAM and resume it across restarts (
Peer::seed_in,Peer::open,Bootstrap::backend); the in-memoryLocalbackend remains the default and existing spellings (Peer<T>,Rumors<T>) compile unchanged.Architecture (bottom-up)
Kv(public extension point, theLinkof storage): named byte tables, cursor transactions, sync closures under async entry points, a prefix-consistency crash floor, durability delegated to the store's documented policy.conformance::kvvalidates caller implementations, with lying-store negative controls proving each probe's teeth.Memoryis the reference store; its committed-state history makes every prefix a reopenable crash survivor.KvBackend<K>: node records with eager resident child fans and stored memos, store-allocated node IDs (the Merkle hash is position-relative and deliberately not an identity), strong counts + held-pin registrations with deferred GC that keeps every transaction small, pending-node custody so compressed spans persist as one record, and crash recovery that sweeps dead process state on open. Identity (the party clock) lives in the store's meta table, updated atomically with every root flip; party shrinks are recorded before the donation crosses the wire (a committed law asserts the ordering at every crash prefix).commit lock → sync prep → build off the watch → persist → poll-atomic swap publish. Observers wake exactly once per committed change; local sends are never blocked across a wire session; cancellation windows are enumerated and documented onBatch::commit.send/redactawaited,Batch::commitexplicit (drop aborts), owned reads,Stream-shaped iteration,Resultobserver items. Read paths were then recovered to 1.1–1.4× of the sync-era baseline on production delta shapes (measured on reserved cores; tables in the tuning commit messages).Verification story
KvBackend<Memory>againstLocalat both the backend seam and the public API (mixed-backend gossip converges over the unchanged wire — snapshots byte-identical throughout the branch).Review flags
Arc<Version>interning (b5569b03) — pending thebeforeBytes-representation refactor, which would make it unnecessary (it bought walks 8–29% but costs writes 3–5%). The back-out is that commit's reverse alone; the KV schema serializes plainVersion. Decide before merge.Memory's history), with its brokendiffdenylisted by clippy def-path + atools/denylistgate tier (adequacy-verified). The upstream defect report/patch is staged separately in~/src/imbl.retire's future was found over the 1 KiB budget pre-campaign and is now boxed under it; KV gossip futures carry a documented 2×1024 budget (honest handle size, not tower inlining).