_restore_ledger: the reset_owned CAS anchor trusts a post-reset observation and can overwrite a concurrent writer

[pbean]

Follow-up residual from PR #726 (the #286 + #469 ledger/board cross-process
serialization program). Filed rather than fixed there because every candidate remedy
changes a design decision that PR reviewed and pinned, and it surfaced in the
finalization phase of an eight-phase program.

The defect

Engine._restore_ledger (src/bmad_loop/engine.py) gates its pre-harvest restore on two
compare-and-set anchors. The second is, in effect:

reset_owned = (current == observed) and <git owns the ledger>

where observed is read immediately after _rollback_or_pause() returns.

If another process writes a tracked ledger in the window between git reset --hard
completing inside _rollback_or_pause() and that unlocked observed read, then observed
is the rival's text. Later current == observed holds, the branch labels the rival's
write reset_owned, and the restore overwrites it with snapshot — silently dropping the
other process's entries.

Reading sooner only narrows the window; it cannot close it. The anchor is derived from an
observation taken after the event it is meant to attest.

Reported by codex on PR #726 and verified correct against the code:
#726 (comment)

Severity: residual, not a regression

Before #726 this site performed an unconditional overwrite with no anchor at all. The
guarded version is strictly narrower — it now requires a rival to land inside a small
post-reset window rather than at any point in the restore. So #726 is a large net
improvement at this site, and this issue tracks the part it did not close.

Candidate remedies

1. Derive the expected state independently of the post-reset read — e.g. take the
   committed blob from git rather than trusting an observation of the working tree. Costs a
   new subprocess spawn inside a restore path plus its failure modes, and ledger_lock
   deliberately covers file I/O only, so the probe would have to sit outside the lock (the
   same hoist S7 already had to make for the git-ownership probe).
2. Treat the ambiguous state as divergent and refuse to write — journal and escalate
   instead of restoring, which is exactly what _ensure_migration's CAS already does on
   divergence. Cheaper and loud, but it gives up the restore in cases where it would have
   been correct.

Option 2 matches the degrade policy already established elsewhere in the family; option 1
preserves more of the restore's usefulness. The choice is a design call, not a repair.

Constraints any fix must respect

• Locks never span a subprocess, session, or pause (deferred-work.md: concurrent writers can lose entries, duplicate DW ids, and revert closures #286 acceptance criterion) — a lock
  around the reset window is not available as a remedy.
• Exclusion probes use blocking=False + pytest.raises(OSError), never sleeps.
• A restore-write test over a tracked ledger is vacuous: git reset --hard restores the
  file regardless of whether the restore code runs. Any test for this must use an untracked
  fixture or assert on the journal, a point proven by control in session 6 of the program.

Related: #726, #286, #469.

[pbean]

Planning complete (2026-08-26, session-program). Decisions:

• Remedy: blob-derived anchor. The reset_owned write arm will compare against the committed baseline blob (verify.worktree_file_bytes_at_revision at task.baseline_commit, decoded + universal-newline normalized), probed outside the lock. Probe failure degrades to no anchor — uncertainty never authorizes a write (the inverse of _ledger_is_gits_to_restore's keep-the-file direction). New journal kind: ledger-baseline-probe-failed.
• Scope: all three sites. The same trust-a-post-reset-observation shape exists in _restore_defer_ledger (write arm, engine.py) and SweepEngine._ensure_migration (sweep.py); both adopt the anchor. Divergence keeps each site's existing outcome (journaled skip / append-only merge / escalation). For the sweep's untracked ledger the expectation is the rejected rewrite the attempt itself graded.
• Governing principle: a post-reset observation may justify a SKIP, never a WRITE.

Lands with #736 as one PR off main (post-#726) on branch pbean/noop-lock-blob-anchor-735-736; phases 3–4 of the 5-phase plan cover this issue. Plan: ~/.claude/plans/mighty-bubbling-badger.md (local).

[pbean]

Session 3/5 of the #735+#736 program — _restore_ledger now anchors on the committed blob.

Branch pbean/noop-lock-blob-anchor-735-736. A single PR at the end of the program
lands both halves of the family; this issue stays open until then.

What landed (src/bmad_loop/engine.py):

• New _ledger_rel() — the repo-relative derivation extracted out of
  _ledger_is_gits_to_restore, answering (rel, None) / (None, fault) /
  (None, None) and never raising. The fault answer is deliberately left
  undecided there, because the two consumers degrade in opposite directions.
• New _ledger_baseline_text(task) — reads the ledger's committed blob at
  task.baseline_commit via verify.worktree_file_bytes_at_revision, which is
  exactly what reset --hard republished. Answers (True, text),
  (True, None) (determinate absence at baseline) or (False, None) (no
  baseline, no rel, or a probe fault). Faults are caught inside the helper,
  journal the new kind ledger-baseline-probe-failed, and withhold the anchor.
• _ledger_is_gits_to_restore rebased onto _ledger_rel; observable behavior
  byte-identical, and its five existing tests pass untouched.
• _restore_ledger: the probe runs pre-lock (it spawns git, and ledger_lock
  covers file I/O only, per deferred-work.md: concurrent writers can lose entries, duplicate DW ids, and revert closures #286), and reset_owned = current == observed and gits
  becomes reset_owned = anchored and current == expected.

The design rule this encodes: a post-reset observation may justify a SKIP,
never a WRITE. Every skip arm still reads the working tree; the one write arm now
derives its anchor from git. Note the fault direction is inverted from the gits
probe — that one degrades True because its consumer is an unlink and uncertainty
must never delete; this one degrades to no-anchor because its consumer is a write
and uncertainty must never write.

Chose remedy 1 from the issue, with remedy 2's degrade as the failure mode: an
unprovable baseline falls through to the journaled skip rather than writing.

Tests — 8 new in tests/test_engine.py, all 8 ablation-proven singly:

• the e2e defect proof: a rival appended to a tracked ledger between
  _rollback_or_pause returning and the restore's observation read now survives,
  with one ledger-restore-skipped-diverged row. Reverting the one anchor line
  reproduces the loss verbatim — the ledger comes back holding only the snapshot.
  Per the issue's own constraint, the oracle is the rival's survival and the
  journal kind, never the restored bytes.
• a positive control that the reset-owned write arm still fires (hardcoding
  reset_owned = False reds it) — the row that would catch a silently
  never-true anchor.
• a direction pin: probe fault ⇒ no write, both journal rows, nothing raised.
• a CRLF-blob row: worktree_file_bytes_at_revision applies working-tree filters,
  so under core.autocrlf=true the blob is CRLF while _ledger_text's read_text
  has already normalized the file to LF. Without normalization reset_owned would
  be silently never-true on Windows; this row makes that Linux-visible.

Suite green (pyright 0, trunk check --all clean). Sessions 4 and 5 carry the
same anchor to _restore_defer_ledger and SweepEngine._ensure_migration, then
open the PR.

[pbean]

Progress — session 4/5: the remaining two write arms now anchor on the committed blob (bf34d389, branch pbean/noop-lock-blob-anchor-735-736)

Session 3 landed _ledger_baseline_text and re-anchored _restore_ledger (the site this issue filed). This session adopts that anchor at the other two arms exploration found sharing the shape:

• Engine._restore_defer_ledger — the reset-owned write arm was current == observed; it is now anchored and current == expected, with expected derived from the ledger's committed blob at task.baseline_commit, probed outside the lock. The post-reset observation is kept, but only for the skip it already served. No anchor degrades to the existing merge, which is append-only and so cannot destroy a rival's write.
• SweepEngine._ensure_migration — same substitution, plus the read at the top of the attempt is split so absence survives: rewrite (str | None) keeps the difference between "the session emptied the ledger" and "the session deleted it", while new_text stays str for validate_migration. On a tracked ledger the anchor is the blob; on an untracked one (no blob, and reset --hard cannot have touched it either) it is the rejected rewrite this attempt actually graded — down to None == None. A probe fault or a missing baseline yields no anchor and escalates; there is no append-only merge to degrade into here, so the escalation is the degrade. The now-dead observed read was deleted rather than left to invite a second comparison against it.

Behavior delta to disclose: a probe fault in the sweep now pauses the sweep with the existing "changed underneath the failed migration attempt" escalation where it previously wrote on the observation alone — strictly fewer writes, and the resume path (attempt budget reset + re-read) is the right recovery for a rival-corrupted migration input.

Tests: 6995 → 7000 passed / 49 skipped, pyright 0 errors, trunk check --all --no-fix clean. Five new tests (rival-inside-the-reset-window for the defer restore and for both the tracked and untracked migration restore; probe-fault direction pins for both sites), plus the defer positive control extended with a no-divergence row so a never-true anchor cannot stay green behind the merge.

Ablations — 7, run singly, each reddening its intended oracle (not just the rc):

Ablation	Oracle that reddened
defer arm → current == observed	KeyError: 'DW-2' — the rival's entry is gone
defer anchor hardcoded False	the no-divergence row fires
probe fault returns (True, self._ledger_text()) (defer)	the diverged row is absent — it wrote instead of merging
migration arm → current == observed (tracked)	escalation is the attempt-cap one, not "changed underneath"
migration arm → current == observed (untracked)	same
probe fault inverted (migration)	same
migration anchor hardcoded False	the untracked positive control's restored bytes red — the write arm is live

Two stale ablation records that named the now-deleted current == observed expression were refreshed to name the new arm (tests/test_engine.py, tests/test_sweep.py); no assertion changed.

Session 5 (docs, CHANGELOG, completeness audit, PR) remains.

[pbean]

Session 5/5 — docs, audits and the PR. All three sites are fixed and up for review in #737 (ready for review; it closes this issue and #736 on merge — not merged, that is the maintainer's call).

Remedy 1 from the issue, applied to the whole class rather than the one reported site. Each write arm now anchors on the ledger's committed blob at task.baseline_commit, read through the existing verify.worktree_file_bytes_at_revision — exactly what reset --hard republished, and something no rival inside the reset window can have authored. The probe runs outside the lock, as the issue's own constraint requires.

The rule the fix encodes: a post-reset observation may justify a SKIP, never a WRITE. The observation is kept for the skip arms, where declining to act is safe whoever wrote the bytes.

Sites: _restore_ledger (the one filed), _restore_defer_ledger, and SweepEngine._ensure_migration. Each degrades in the only direction its semantics allow when no anchor can be derived — the retraction skips (ledger-restore-skipped-diverged), the defer restore merges by appending what disk has lost (defer-ledger-restore-diverged), and the sweep escalates (sweep-migration-restore-diverged). New helpers _ledger_rel and _ledger_baseline_text, plus one new journal kind ledger-baseline-probe-failed; the blob's newlines are normalized to LF, without which the anchor would be silently never-true on Windows.

Note on the degrade direction, since it is the one thing easy to get backwards: _ledger_baseline_text degrades to no anchor, the inverse of the gits probe, which degrades to True. The gits probe feeds an unlink and uncertainty must never delete; this one feeds a write arm and uncertainty must never write. Copying the other probe's degrade here would reopen this issue through the repair itself.

Released-behavior deltas disclosed in the PR: the doubly-degraded path (gits probe fault plus blob probe fault) previously could still write and now skips, merges or escalates; a sweep probe fault now pauses with the existing "changed underneath" escalation. The ABA residual stays residual and is documented, not claimed fixed — a rival whose text is byte-equal to the committed blob (or, on the sweep's untracked ledger, to the rejected rewrite) is indistinguishable in principle.

Tests grade the rival's survival, the journal kind and the escalation reason rather than restored bytes, since a restore-write assertion over a tracked ledger is vacuous — reset --hard puts the file back regardless. Suite 6954 → 7000 passed / 49 skipped, pyright 0, trunk check --all --no-fix clean.