CDEB research line: v6 is TERMINAL_HOLD_FINAL and a successor needs an owner decision

[MongLong0214]

Next action: v8 is calibrating its judge panel against 47 controls. Nothing is blocked. v7 closed as TERMINAL_HOLD_FINAL and is merged; v8 is owner-authorized and runs a blinded three-judge panel so that an unresolved machine boundary no longer stops the study.

This issue tracks the CDEB research line to its end.

v7 — terminal

CDEB-Fresh v7 reached TERMINAL_HOLD_FINAL before any product-effect episode.
Eight of the fixed 17 decisions yielded a semantic boundary precise enough for
deterministic oracle construction and nine did not. Because the preregistered
population was fixed at all 17 tasks and unresolved ambiguity was terminal under
v7, the population was not reduced post hoc. This result concerns deterministic
machine adjudicability, not the causal effect of CommitLore delivery.

fixed benchmark population           17
semantic boundary settled             8
semantic boundary unresolved          9
measured product-effect episodes      0

Each decision was read twice by independent sessions seeing only the rule and the
frozen repository. Where both drew a boundary, a third tried to build a tree they
would classify differently — agreement is what survived that. Where they split, a
fourth read the rule again with both attempts anonymised.

The nine that did not settle fail the same way: a rule turning on a term it never
defines, with the recorded reason reaching further than the recorded words —
literally, a badge, add the two paths, prose field. One is not vagueness
at all: hand-maintained is about how a file came to exist, and a finished tree
does not record that.

These decisions were written by people for people in a commit trailer, and they
read perfectly well that way.

Merged as #856.

v8 — owner authorized, running

The owner supplied a new SSOT rather than accepting that the line ends. v8 keeps
all 17 tasks and changes the instrument:

	v7	v8
primary instrument	deterministic oracle per candidate	blinded three-judge semantic panel
unresolved boundary	terminal	neither exclusion nor hold reason
scale	340 episodes, never reached	340 episodes × 3 judges = 1,020 judgements

The exact study question: across the exact 17 frozen decision-sensitive
tasks, does automatic delivery of the candidate-relevant CommitLore decision
before the first relevant mutation raise the rate of episodes that complete, pass
both acceptances, and are read as compliant by a blind panel — relative to
suppressing that automatic delivery?

Judges never see the arm, the boundary status, the agent identity, the transcript,
the delivery payload, the acceptance result, or the v7 specifications. That last
exclusion matters: a judge handed a boundary specification would apply it rather
than read the decision, which would make the panel a proxy for the oracle v7
could not build.

The panel is the instrument, so its own reliability is published whatever it
shows. If agreement is poor the causal estimates are still computed and released,
but the result cannot be positive or null in a strong sense.

One deviation is already recorded. The SSOT builds the calibration key from 51
controls assuming every Good control carries a known compliant label. v6 kept no
control bytes, so those are the 34 v7 rebuilt — and four of them have no agreed
label, one having drawn a violation verdict from a blind session reading a builder
that was never told the decision. Calibrating on those would select judges that
agree with a disputed key. The key is 47; the four are retained, not deleted.

No automatic v9

v8 is the final planned study of this line. Any successor requires a separate
owner decision and is not generated automatically.

This issue closes when v8 publishes its result — positive, qualified, null,
negative, indeterminate, or a terminal hold.

[MongLong0214]

The owner supplied a successor SSOT — CDEB-Fresh v7, a fixed-benchmark effect trial over the exact 17 v6 TASK_BUILDABLE tasks: 17 × 2 arms × 10 repetitions = 340 measured episodes, no sample-size floor, terminal on any outcome. That answers the decision this issue was tracking.

Reviewing it against the repository, most of it checks out:

claim	result
the 17 candidate IDs	exact match with buildability/summary.json, 8 AOS / 9 gitseed
task, freeze entry, judgement, Bad A for all 17	17/17 present
decision_audit_anchor per candidate (needed for structured target suppression)	present, with ruling/reason/scope/lifecycle
snapshot bundles	present, digests match snapshot-lock.json
v6 artifacts unchanged since the audited SHA	0 files differ

Two things block it at the import gate.

1. The revival oracle does not exist. git ls-files finds no oracle anywhere under the v6 study; buildability/oracles/ is an empty untracked directory. What decided violation in v6 was two blind judges, not a mechanical oracle. The v7 PRD treats an oracle as an existing immutable v6 input in five places — the benchmark manifest, control replay, episode step 12, the reuse list, and the re-explanation audit — and the primary endpoint DSFPS = completed AND functional_pass AND revival == false rests on it.

What v6 left instead is a specification for the oracle's validation, in buildability/directed-violation-limitation.json:

An oracle validated only against directed violations can be tuned to how they happen to be written and miss a spontaneous one. […] An oracle that only recognises the Bad control has learned the control rather than the decision.

The repository has no near-miss control and no Bad B, so each candidate carries exactly one violation shape — the directed one. An oracle built now could only be validated the way v6 explicitly said would not be enough. So this is a missing artifact whose validation inputs are also missing, and building both is unscoped work rather than reuse.

2. The pinned product dist digest matches nothing. The v7 PRD carries 318e1661…33133b91. v6 already measured a0c54297…8bc0e528 at that commit, scanned 338 files there for the pinned value and found none, and recorded it as deviation v6-d001. Read literally, the import gate refuses and the study ends before it starts. This one is a one-line correction.

A third point is methodological rather than blocking: the strong claim gate requires the primary 95% CI lower bound above zero, but that interval measures only execution variability with the 17 tasks fixed. If the pinned agent is near-deterministic the interval collapses toward zero width, and the condition passes on essentially no evidence about generalization — while the task-population sensitivity interval, which would carry that evidence, is explicitly excluded from the gate.

Keeping this issue open until the oracle question is settled, since it decides whether v7 is executable as written.

[MongLong0214]

Correcting the first point above. I wrote that the revival oracle does not exist. That is wrong: one exists, and the validation machinery for oracles exists too. The accurate count is 1 of 17.

bench/cdeb/freeze/oracle-v5.ts                                  gate G2, oracle validation
bench/cdeb/studies/cdeb-fresh-v5/stage1-r1/oracles/
  v4-377f04276465b59d/oracle.py                                 the one oracle — and this
                                                                candidate is one of the 17

oracle-v5.ts defines what an oracle may read (final_tree, functional_acceptance_result), what it may not, and requires discrimination rather than determinism: it must answer differently for a patch that honours the decision and one that revives what the decision ruled out, where both pass functional acceptance.

I also found this gap already named in the repository, in that file's header:

Stage 0's G5 asked reviewers whether a deterministic oracle could be written and 62 candidates passed. None was written. That gap is the single largest thing standing between the design and a measurement, because an oracle that always answers "no revival" is perfectly deterministic and perfectly useless — it would report the treatment working in every arm.

So this is not a new finding. It is a known, documented, load-bearing gap that the v7 PRD does not acknowledge — it classes the oracle among the immutable v6 inputs to reuse.

The conclusion does not change, and the correction sharpens what is missing:

	state
oracle validation infrastructure (G2)	present
oracles for the remaining 16 candidates	absent
differently-shaped violations to red-team each oracle against	absent — one directed Bad A per candidate

The one existing oracle illustrates why the second row matters. It is a thirty-line regex over a fixed list of six paths (.github/workflows, pyproject.toml, setup.cfg, tox.ini, README.md, Makefile) looking for coverage gating. The same forbidden behaviour reached through noxfile.py, a pre-commit config, or a helper script returns false. That is exactly what directed-violation-limitation.json warned about: an oracle that only recognises the Bad control has learned the control rather than the decision.

Having the infrastructure is good news — the spec and the forbidden inputs are already settled, so this does not start from nothing. But writing 16 oracles, constructing a differently-shaped violation for each, and passing G2 is work the v7 PRD does not register anywhere.

The second blocker in my earlier comment — the pinned dist digest matching no artifact — is unaffected and still stands.

[MongLong0214]

One more correction, to the bundle point in my first comment. I flagged the snapshot bundles being outside version control as an operational risk. It is not an oversight — it is a decision this repository recorded at the time, in r-v3sealedcensus:

Ruled out: committing the bundles so the digest and the bytes travel together — they are large binaries, and the digest with a refusal on mismatch gives the same guarantee without putting them in every clone.

That reasoning holds, and the digests in snapshot-lock.json do verify integrity. My concern was availability rather than integrity — a multi-day trial reads from two files (2.1 MB combined for the pair v7 needs) that no clone carries — but that is a property of a decision already taken with its reason written down, not a defect to raise against the v7 PRD.

Net, the review of the v7 PRD comes down to two things:

1. Oracles exist for 1 of the 17. The validation gate (oracle-v5.ts, G2) is in place and settles what an oracle may read and what it must discriminate, so this does not start from nothing. But sixteen oracles, plus a differently-shaped violation to red-team each against, is work the PRD registers nowhere — it classes the oracle among the immutable inputs to reuse.

2. The pinned dist digest matches no artifact at the commit it pins. A one-line correction, already documented as deviation v6-d001.

The methodological point about the claim gate leaning on an interval that measures only execution variability stands as written.