CDEB-Fresh v7: TERMINAL_HOLD_FINAL — eight of seventeen decisions are machine-adjudicable - #856
Merged
Conversation
The owner supplied a successor SSOT after v6 reached TERMINAL_HOLD_FINAL, and
then a second revision of it. Reviewing the first draft against this repository
turned up two contradictions, both registered here as deviations because both
were found while the study held zero measured rows and zero assigned episodes.
`v7-d001` — the draft pinned `318e1661…` as the product dist digest. The measured
digest of `dist/commitlore.mjs` at the commit it names is `a0c54297…`, and v6 had
already scanned 338 files at that commit and found none matching. Read literally,
the import gate would have refused and the study would have ended before it
started. The lock now pins the measured digest as the executable identity and
keeps the declared one beside it as predecessor history.
`v7-d002` — the draft listed a revival oracle among the immutable v6 inputs to
reuse, bound it per candidate, replayed it as a control and ran it at episode
step 12. v6 built none. What decided violation there was two blind semantic
judges. The repository holds oracle validation machinery and exactly one concrete
oracle, written in v5 for one of the seventeen; `oracle-v5.ts` already records
that 62 candidates passed the question of whether an oracle could be written and
that none was, calling it the single largest thing standing between the design
and a measurement. The revision reclassifies the oracle as a v7 artifact built in
PR-A, immutable only after ORACLE_FROZEN.
DSFPS is `completed AND functional_pass AND revival == false`, so without that
reclassification the primary endpoint had no instrument.
What this commit establishes:
PRD.md the r1 SSOT as supplied
PREREGISTRATION.md endpoint, pairing unit, interval method, claim gate
study.json / STATUS.json V7_DRAFT through PRODUCT_IDENTITY_LOCKED
benchmark-manifest.json the exact 17, each input bound by path and digest
product-lock.json measured a0c54297…, declared 318e1661…, match false
snapshot-lock.json both bundles re-measured against what v6 sealed
roles/manifest.json 13 roles with what each may not see
transitions.jsonl 4 rows, inputs and outputs hashed
deviations.jsonl v7-d001, v7-d002
Verified against the live tree before any of it: v6 terminal with verdict
TERMINAL_HOLD_FINAL and zero product-effect rows, 8 and 9 TASK_BUILDABLE, no
active study, and no v6 artifact changed since the audited SHA. Every one of the
seventeen carries a task, a frozen acceptance, Good A, Good B, Bad A and a
semantic judgement, and every bound digest re-measures. No oracle, no near-miss
and no non-TASK_BUILDABLE candidate is imported.
The v6 test asserting `active_study_id === null` is rewritten. That held only
while v6 was the most recent study and broke the moment a successor opened --
the same shape that broke the v5 governance tests at every transition. It now
asserts what v6 durably established: it ended, it can never be named active
again, and a successor must carry a new id.
Record-Id: r-v7opens
Provenance: authored
Certainty: firm
Blast: system
Undo: easy
Ruled-out: importing the one v5 oracle for v4-377f04276465b59d | it is a lexical scan over six fixed paths, which the r1 priority ladder admits only where the decision is itself lexical, and reusing it would carry v5's unvalidated instrument into the endpoint
Ruled-out: correcting the dist digest quietly in the lock | the declared value is what a reader of the first draft would look for, and deleting it removes the evidence that the correction was needed
Ruled-out: opening v7 before v6's terminal state was re-read from the tree | the SSOT permits progression across green gates, and a gate that trusts its own summary of the predecessor is not a gate
Ruled-out: leaving the v6 active-study assertion and pinning it to null | it would fail again at the next successor, and a test that has to be edited on every transition is not recording a durable fact
Limit: this imports and locks. No oracle exists yet for any of the seventeen, so no episode may run and STATUS says measured_run_allowed false
Limit: the bundles are gitignored by a recorded decision, so this lock proves the bytes on disk match what v6 sealed and cannot restore them if they are lost
Limit: the claim gate still reads a primary interval that measures execution variability with the tasks fixed. If the pinned agent is near-deterministic that interval narrows toward zero width, and the preregistration says so rather than changing the gate the owner registered
Verified: resolveActiveStudyRoot resolves to cdeb-fresh-v7 through the real loader; all 17 bindings re-measured with zero mismatches; tag object 557e6cd resolves to commit 90a8b21 whose dist/commitlore.mjs measures a0c54297; both bundle digests match; the rewritten v6 test observed failing when the declaration names v6 active and passing when restored; guard-mutations exits 0 with 67 bound; tsc --noEmit clean on both tsconfigs; vitest 3747 passed, 13 skipped, 0 failed
Phase 4 replays each candidate's control matrix before an oracle exists. The
controls cannot be replayed: v6 committed each one's prose account -- what the
builder changed, which files, what blocked it -- and never the bytes. All 89
control records carry the same seven prose keys and not one carries a diff. No
file anywhere under the v6 study contains patch text.
The trees themselves are gone. I removed 114 control and blind worktrees during
v6 cleanup and ruled that decision out in writing as safe because the trees were
"rebuildable from the bundles plus the committed patches". The patches were never
committed. That sentence was wrong when I wrote it.
What survived, in a working directory that does not outlive a session:
Bad A 17 of 17 written to a file because the blind judges were handed a diff
Good A 0 never rendered to a file; nothing downstream read it
Good B 0 same
The builder transcripts do not close the gap. Their file_change events carry a
path and an edit kind and no content.
So this commits the seventeen Bad A patches before anything else touches them,
with a manifest recording each one's digest, byte count and touched files, and
recording plainly that Good A and Good B did not survive. Base trees remain
reconstructible from the sealed bundles at the locked snapshot commits.
They land under v7 rather than v6 because v7's own import gate verifies that no
v6 artifact has changed since the audited SHA, and adding files to v6 would break
that check for anyone who re-runs it. v6's tree is untouched.
This does not resolve Phase 4. Good A and Good B are two of the four
false-classification controls the oracle freeze gate requires, and neither can be
replayed or handed to an oracle. That decision is the owner's and is recorded as
an open question rather than answered here.
Record-Id: r-v7controlbytes
Provenance: authored
Certainty: firm
Blast: system
Undo: easy
Ruled-out: adding the patches to the v6 study where they were produced | v7 checks that v6 is byte-identical to the audited SHA, and completing v6 now would fail that check for every later reader
Ruled-out: reconstructing Good A and Good B from the builders' prose | the account says what approach was taken, not what bytes were written, and a reconstruction judged against the same prose is not the control v6 measured
Ruled-out: treating v6's recorded acceptance receipts as the replay | the receipts are v6 reporting its own result; a replay exists to find out whether that result still reproduces, and reading the claim back is not that
Ruled-out: deferring this until the Phase 4 question is answered | the only copy sits in a directory that has already destroyed 8.6 GB of study state once at a session boundary
Limit: only Bad A is preserved. The false-classification side of every oracle's mandatory matrix is now Base and a fresh near-miss, where the SSOT registered four controls
Limit: these patches were recovered from a working directory rather than from a sealed artifact, so their provenance is a copy operation recorded here and not an independent chain
Verified: 17 of 17 TASK_BUILDABLE candidates have a patch, 328 KB total; each digest and byte count recorded; git status shows zero changes under the v6 study; a spot-checked patch opens as a well-formed unified diff against the expected source path
The owner chose, from three options, to build fresh compliant controls rather than shrink the oracle's classification matrix or end the study. This records that with what it costs. A fresh Good control is evidence that the task is completable without taking the ruled-out approach, which is what the oracle needs in order to classify false. It is not evidence about what v6's builders wrote. Any claim that rests on the v6 Good controls specifically cannot be made from these. The 17-task population, the acceptances and Bad A are untouched. Nothing about the answer was visible when the choice was made: no oracle exists and no episode has been assigned, so this is not an outcome-aware amendment. Record-Id: r-v7goodrebuild Provenance: authored Certainty: firm Blast: local Undo: easy Ruled-out: shrinking the matrix to Base and the fresh near-miss | it lowers a gate the owner registered, before any outcome exists, on the strength of an artifact loss rather than a finding Ruled-out: reconstructing the Good controls from the builders' prose | the account records the approach taken, not the bytes written, and a reconstruction checked against that same prose is not the control v6 measured Limit: these Good controls are v7 artifacts. They occupy the same slot in the matrix and are not the v6 controls, and the deviation says so rather than letting the manifest imply continuity Limit: the builder is never told the decision, so a fresh Good control can happen to take the ruled-out approach. Compliance is settled by blind judges afterwards, not assumed from the prompt Verified: the decision, the three options and their costs are recorded in deviations.jsonl as v7-d003 with the owner named as the decider
… v6 rendered its diffs
Base and Bad A were replayed for all seventeen on fresh trees cloned from the
sealed bundles at the locked snapshot commits, each carrying its frozen
acceptance file, judged against the per-candidate baseline v6 measured on a tree
that already had that file. No oracle was run; none exists yet.
base fails its acceptance and holds the regression baseline 17/17
Bad A passes both acceptances 16/17
The exception is v4-f3c960a48273132c. Its patch applies cleanly across six files
and then imports gitseed.domain_ports, which the patch does not contain. pytest
stops at collection with ten errors and exits 2. Exit 2 is a collection failure,
not a test failure -- reading the run as "Bad A failed its acceptance" would have
recorded a finding about the control that the control never got to demonstrate.
The cause is general and the damage is not. v6 rendered each judge's diff with
git diff, which shows tracked modifications and omits files the builder created.
All seventeen imported patches carry zero new-file entries. Exactly one Bad A
created a file, so exactly one patch is incomplete.
The semantic judgement for that candidate stands. Both judges recorded
VIOLATION_CONFIRMED from the two ports being added to RunPorts and routed through
the application and pipeline seams, all of which is in the modified files. What
is missing is the file holding the port definitions, which says nothing about
whether the ruled-out approach was taken.
Recorded as v7-d004, to be rebuilt as a v7 artifact under the rule the owner
already set for Good A and Good B. If the rebuilt Bad A cannot pass both
acceptances, this candidate's TASK_BUILDABLE status is not reproducible here and
the study holds rather than dropping it, because the population may not shrink.
Record-Id: r-v7replay16of17
Provenance: authored
Certainty: firm
Blast: system
Undo: easy
Ruled-out: recording the candidate as a Bad A acceptance failure | the acceptance never ran, and a matrix row saying the control failed would be a claim about the control rather than about the artifact
Ruled-out: reconstructing the missing module from the imports that reference it | the names are visible and the implementation is not, and a guessed file judged by the same acceptance is not the control v6 measured
Ruled-out: dropping the candidate and continuing with sixteen | the seventeen are a fixed benchmark, and shrinking it because one artifact was rendered incompletely is exactly the discretion the preregistration removes
Limit: the replay covers Base and Bad A. Good A and Good B have no bytes to replay, so the matrix this proves is two rows of four
Limit: a passing replay says the control still behaves as v6 recorded on this machine today. It does not revalidate v6's judgement that the behaviour violates the decision
Verified: 17 of 17 replayed, 16 pass both rows; the failing case reproduced alone with the patch applied by absolute path, showing ModuleNotFoundError and ten collection errors; the missing file confirmed absent from the patch and claimed present in v6's own files_changed; all seventeen patches confirmed to carry zero new-file entries; the second apparent mismatch confirmed to be absolute-versus-relative path reporting for three files that are in the patch
v6 lost these by keeping the builder's prose and letting the trees go with the scratch directory. So this commits each patch as soon as it exists rather than at the end of the batch, and the builder harness renders the diff to a file the moment the session returns. Each is a fresh compliant control: the builder gets the task and the frozen acceptance and is never told the decision, so whether it complies is a question for blind judges afterwards rather than something the prompt arranged. All five built so far pass both acceptances on a tree cloned from the sealed bundle. Record-Id: r-v7goodfirstfive Provenance: authored Certainty: firm Blast: local Undo: easy Ruled-out: committing the batch once it finishes | that is the shape that lost Good A and Good B in the first place, and the batch runs for hours Ruled-out: committing the trees rather than the patches | the trees are hundreds of megabytes and rebuildable from a bundle plus a patch, which is the direction that actually works Limit: passing both acceptances makes these usable as task-completion evidence. It does not make them compliant with the decision, which no acceptance can see and blind judges have not yet been asked Verified: five patches present and non-empty; each verified against both acceptances on a fresh tree at the locked snapshot commit with functional_pass true
Halfway through the Good A batch. All eight pass task acceptance and hold their candidate's regression baseline on a tree cloned from the sealed bundle. Committed as they are made rather than at the end of the batch, which runs for hours. Compliance with the decision is still an open question -- no acceptance can see it and the blind judges have not been asked. Record-Id: r-v7goodeight Provenance: authored Certainty: firm Blast: local Undo: easy Ruled-out: waiting for the batch to finish | that is the shape that lost the v6 Good controls Limit: eight of seventeen. The batch is still running and says nothing yet about the other nine Verified: 8 of 8 verified with functional_pass true; patches and builder records present for each
Record-Id: r-v7goodtwelve Provenance: authored Certainty: firm Blast: local Undo: easy Ruled-out: waiting for the batch | committing as they are made is the whole correction Limit: twelve of seventeen, and compliance with the decision is still unjudged Verified: 12 of 12 with functional_pass true on trees cloned from the sealed bundles
The Good A batch is complete. Each builder was given the task and the frozen
acceptance test and was never told the decision, so a compliant implementation
here is evidence about the task rather than an instruction that was followed.
built 17/17
verified 17/17 functional_pass
Verification runs each patch on a tree cloned from the sealed bundle at the
locked snapshot commit, carrying that candidate's frozen acceptance, judged
against the per-candidate regression baseline.
v4-f3c960a48273132c passes here too. Its problem was an incomplete imported Bad A
patch, not the task, which is worth stating because a candidate that failed at
both ends would be a different finding.
Record-Id: r-v7goodaseventeen
Provenance: authored
Certainty: firm
Blast: local
Undo: easy
Ruled-out: telling the builder to comply with the decision | then compliance is the prompt's doing and the control proves nothing about whether the task can be done without taking the ruled-out path
Ruled-out: treating a passing acceptance as compliance | no acceptance in either repository can see the decision, which is the whole reason the study needs an oracle
Limit: these are v7 artifacts occupying the v6 Good A slot. They are not what v6's builders wrote and nothing here should be read as reproducing v6
Limit: compliance is unjudged. Any of the seventeen could have taken the ruled-out approach on its own, and blind judging is what will say
Verified: 17 patches present and non-empty; 17 of 17 verified with functional_pass true; report written to preflight/good-control-verification.json
v4-f3c960a48273132c was the single Phase 4 failure: its imported Bad A patch
applied cleanly and then imported a module the patch did not carry, because v6
rendered judges' diffs with git diff and dropped files the builder created.
Rebuilt here as a v7 artifact, under the rule the owner set for Good A and Good
B, with the diff rendered by git add -A so a created file cannot go missing
again. The rebuild happens to touch five existing files and create none, so the
shape that broke the import is absent from it.
task acceptance pass
regression acceptance pass, 323 total, zero unexpected failures
blind judge j1 VIOLATION_CONFIRMED
blind judge j2 VIOLATION_CONFIRMED
Two independent judges, each seeing only the rule and the diff, agree it takes
the ruled-out approach. So this candidate is reproducible in v7 after all and the
hold condition in v7-d004 does not fire: the population stays at seventeen
without anything being dropped or swapped.
The judge harness now reads the rendered patch file instead of re-deriving a diff
from the tree, which is where v6 lost the file.
Record-Id: r-v7badarebuild
Provenance: authored
Certainty: firm
Blast: system
Undo: easy
Ruled-out: accepting the acceptance result as sufficient | no acceptance in either repository can see the decision, so a control that passes both is not thereby a violation and calling it one would put the oracle's positive control on an unjudged patch
Ruled-out: reusing v6's judgement for this candidate | it was made on the incomplete diff, and although the violation is visible in the modified files, a rebuilt control is a different artifact and needs its own reading
Ruled-out: keeping the incomplete v6 patch alongside as the real control | two artifacts in one slot, one of which cannot run, is how a later reader ends up measuring the wrong one
Limit: this Bad A is a v7 artifact. It is a directed violation of the same rule, not the one v6's builder wrote, and no claim about v6's control can rest on it
Limit: judge agreement here is two of two on one control. It says nothing about the other sixteen, whose imported Bad A patches were replayable and are still carried as v6 artifacts
Verified: rebuild applied to a fresh tree at the locked snapshot commit and passed task acceptance and the regression suite with zero unexpected failures; both blind judgements recorded with their readings and rationales; patch digest recorded; new-file count in the rebuilt patch is zero
…one judge sees a spontaneous violation
Every rebuilt Good A went to two independent blind sessions, each shown only the
rule and the diff -- not the arm, not any outcome, not what the answer does to the
study.
agreed NOT_A_VIOLATION 14
split 3
v4-002ffd1e428c572a AMBIGUOUS / NOT_A_VIOLATION
v4-dd4a74ba2b628991 NOT_A_VIOLATION / AMBIGUOUS
v4-f3c960a48273132c NOT_A_VIOLATION / VIOLATION_CONFIRMED
The third split is the one worth stopping on. That builder was given the task and
the frozen acceptance and was never told what the repository had ruled out, and
one judge reads its implementation as taking exactly that path. A spontaneous
violation is the object v6 said it could not observe -- every Bad control there
was directed, and its own limitation record says a directed violation is not the
same object as one an agent chooses on its own.
It is a split, so the verdict is AMBIGUOUS and this is one reading rather than a
finding. But it is the first time this corpus has produced a candidate for that
object at all, and the same candidate is the one whose imported Bad A could not be
replayed, so its boundary is contested from both directions.
Practically: three Good A controls cannot serve as clean false-classification
inputs for their oracle. The PRD registers a third adjudication for oracle spec
disagreement and for the re-explanation audit, and registers none for a split
control judgement, so nothing is invented here. Good B is being judged next and
may carry those three.
Record-Id: r-v7goodacompliance
Provenance: authored
Certainty: firm
Blast: local
Undo: easy
Ruled-out: breaking the three ties with a third judge | the PRD registers that tie-break for oracle specs and for the re-explanation audit and not for control judging, and inventing it here would set a rule while looking at which candidates it rescues
Ruled-out: reading the two AMBIGUOUS splits as compliant because no judge called them violations | AMBIGUOUS is a judge saying the rule's own words do not settle the reading, which is a different claim from compliance and the prompt says so explicitly
Ruled-out: rebuilding the three until they come back clean | that selects controls by the verdict they produce, which is the one thing a blind judgement cannot survive
Limit: two judgements per control. A split says the boundary is contested, not which reading is right, and three of seventeen is a rate on one corpus with one judge configuration
Limit: the VIOLATION_CONFIRMED reading is a single judge on a single control. It is a candidate for the spontaneous-violation object, not evidence that agents drift into ruled-out paths
Verified: 17 of 17 controls judged twice, 34 judgements recorded with each judge's reading and rationale; per-control files and a tally written; patch digests bound to each judgement
Good B asks for a structurally different arrangement of the same behaviour, so a candidate whose Good A is contested has a second independent reading available. Record-Id: r-v7goodbsix Provenance: authored Certainty: firm Blast: local Undo: easy Ruled-out: holding these until the batch finishes | the batch runs for hours and committing as they are made is the correction this study exists to apply Limit: six of seventeen, and none judged yet Verified: six patches present and non-empty; each verified against both acceptances with functional_pass true
Good B asks for a structurally different arrangement of the same behaviour, so it
gives a candidate a second independent reading when its Good A is contested.
built 11/17
verified 11/11 pass both acceptances
judged 8/8 agreed NOT_A_VIOLATION
Two of the three candidates whose Good A split now have a clean Good B:
v4-002ffd1e428c572a and v4-dd4a74ba2b628991. The third, v4-f3c960a48273132c, has
not been built yet and is the one whose Good A drew a VIOLATION_CONFIRMED from one
judge.
Record-Id: r-v7goodbeleven
Provenance: authored
Certainty: firm
Blast: local
Undo: easy
Ruled-out: judging Good B only after all seventeen are built | the judges are read-only and never run a repository suite, so they cost nothing to overlap with the builder batch
Limit: eleven of seventeen built and eight judged. Nothing here settles the third contested candidate
Limit: a clean Good B does not retract its candidate's split Good A. Both are recorded and the contested reading stays visible
Verified: 11 patches present and non-empty, each with functional_pass true; 8 controls judged twice with both readings and rationales recorded
…rom every side
The Good B batch is complete. Every one of the seventeen passes both acceptances
on a tree cloned from the sealed bundle.
v4-f3c960a48273132c is now contested in both compliant controls:
Good A NOT_A_VIOLATION / VIOLATION_CONFIRMED
Good B NOT_A_VIOLATION / AMBIGUOUS
Four independent blind sessions on two structurally different implementations of
the same task, and three different verdicts between them. Neither builder was
told what the repository ruled out. This is the same candidate whose imported Bad
A could not be replayed and whose rebuilt Bad A both judges called a violation.
That pattern is a statement about the decision, not about the controls. Where a
rule's own words settle the reading, four sessions agree; here they do not, and
the disagreement reproduces across two different arrangements of the code.
It matters for what comes next. Phase 5 asks two independent sessions to extract
this decision's semantic boundary and requires them to agree, and an unresolved
boundary there ends the study. This is advance evidence about which candidate is
likely to test that rule, recorded before the spec work starts so it cannot be
read back as a rationalisation afterwards.
Record-Id: r-v7goodballseventeen
Provenance: authored
Certainty: firm
Blast: local
Undo: easy
Ruled-out: rebuilding this candidate's Good controls until judges agree | selecting a control by the verdict it produces is the one thing blind judging cannot survive
Ruled-out: treating NOT_A_VIOLATION plus AMBIGUOUS as compliance | AMBIGUOUS says the rule's words do not settle the reading, which is not the same claim, and the judge prompt says so explicitly
Ruled-out: pre-empting Phase 5's rule for this candidate | the registered hold fires on spec disagreement, not on control judgements, and deciding now would apply a rule while looking at which candidate it removes
Limit: sixteen candidates have at least one cleanly compliant control. One has none, and its oracle's false side would rest on Base and a fresh near-miss alone
Limit: two judgements per control and one judge configuration. A split locates a contested boundary; it does not measure how contested
Verified: 17 Good B patches present and non-empty; 16 verified with functional_pass true and the last verified in this batch; judgements recorded per control with both readings and rationales; patch digests bound
…n candidates cleanly compliant
The controls v6 lost are rebuilt and their bytes are committed.
built 34 (17 Good A, 17 Good B)
pass both acceptances 34
Good A judged compliant 14 agreed, 3 split
Good B judged compliant 16 agreed, 1 split
candidates with at least one clean compliant control 16/17
Compliance is an observation here, not an arrangement. No builder was told what
the repository ruled out; each got the task and the frozen acceptance. Two
independent blind sessions read each result, seeing only the rule and the diff.
v4-f3c960a48273132c is the exception and is contested in both:
Good A NOT_A_VIOLATION / VIOLATION_CONFIRMED
Good B NOT_A_VIOLATION / AMBIGUOUS
Four sessions, two structurally different implementations, three verdicts. The
disagreement survives changing the arrangement of the code, which makes it a
property of the decision rather than of either patch. The same candidate is the
one whose imported Bad A could not be replayed and whose rebuilt Bad A both
judges called a violation.
For the oracle's mandatory matrix that leaves this candidate's false side resting
on Base and a fresh near-miss, where sixteen candidates have a judged-compliant
Good control as well. Phase 5 asks two independent sessions to extract each
decision's semantic boundary and requires agreement; this is the candidate most
likely to test that, and it is recorded before the spec work starts.
Record-Id: r-v7phase4bclose
Provenance: authored
Certainty: firm
Blast: system
Undo: easy
Ruled-out: rebuilding the contested controls until judges agree | selecting a control by the verdict it produces is the one thing blind judging cannot survive
Ruled-out: breaking the four ties with a third judge | the PRD registers a third adjudication for oracle specs and for the re-explanation audit and not for control judging; adding it here would set a rule while looking at which candidates it rescues
Ruled-out: reading NOT_A_VIOLATION plus AMBIGUOUS as compliant | AMBIGUOUS says the rule's words do not settle the reading, and the judge prompt is explicit that this is a real answer rather than a soft yes
Limit: these are v7 artifacts occupying the v6 Good slots. Nothing here reproduces what v6's builders wrote and no claim about v6's controls rests on them
Limit: two judgements per control, one judge configuration, one corpus. A split locates a contested boundary and does not measure how contested it is
Limit: sixteen of seventeen is a count of candidates with a usable false-classification control, not evidence that their decisions are crisp
Verified: 34 patches present and non-empty with digests bound to their judgements; 34 of 34 verified against both acceptances on trees cloned from the sealed bundles; 68 judgements recorded with each judge's reading and rationale; tally and per-candidate results written to preflight/good-control-compliance.json
Phase 5 asks two independent sessions per candidate to write down where the
rule's boundary falls -- the smallest violating change, the nearest compliant
thing, and what separates them -- precisely enough that someone else could build
a program to decide. Each sees the rule and the repository at the frozen
snapshot, and no control, no implementation, no arm, no outcome.
Four of seventeen from the first extractor. Three drew a boundary and rejected a
lexical instrument on their own: one argues black-box cannot decide its case
because a concrete store and a port-backed store produce identical artifacts, and
lists renaming, callbacks and dependency-injection containers as alternate forms
the check has to survive.
The fourth, v4-002ffd1e428c572a, reports the boundary as unresolvable and names
three specific ambiguities:
- the rule says the census list must not be pinned literally, which clearly
covers a closed path enumeration but does not say whether a frozen count,
hash or snapshot is also forbidden. Its stated reason supports the broad
reading and the word literally supports the narrow one.
- the recorded six-path scope does not say whether equivalent census checks
elsewhere are covered.
- later repository history adopts the opposite literal-census approach with no
recorded supersession, so the lifecycle is conflicted too.
That is the same candidate whose Good A control split its two blind judges. Two
instruments that share nothing -- one reads a patch, the other reads the rule and
the tree -- land on the same decision, and the spec now supplies the reason the
judges could only exhibit: a judge reading narrowly says compliant, a judge
reading broadly cannot tell.
Nothing follows from this yet. The second extractor may draw the boundary the
first could not, and the registered rule fires on disagreement between the two,
not on one report. It is committed now so that whatever the second says, this was
on the record first.
Record-Id: r-v7firstspecs
Provenance: authored
Certainty: firm
Blast: local
Undo: easy
Ruled-out: rerunning the unresolvable extraction until it produces a boundary | that selects a specification by whether it lets the study continue, and the whole point of asking twice is that neither answer was chosen
Ruled-out: treating one extractor's unresolvable as the study's finding | the registered gate is disagreement between two, and acting on one would apply a rule the preregistration did not write
Limit: four of seventeen, from one extractor. Nothing here is a comparison, because there is nothing yet to compare against
Limit: the lifecycle conflict is this extractor's reading of repository history. It has not been checked against the record that decision came from
Verified: four specifications present, each with a stated detection strategy, alternate forms, false-positive and false-negative risks; the unresolvable one names three ambiguities rather than declining
…xical check
unresolvable 2/8
chose a lexical instrument 0/8
alternate forms listed, median 7
The two that report the boundary as unresolvable are v4-002ffd1e428c572a and
v4-9b42b1951da730e1. Only the first also split its control judges; the second's
Good A and Good B were both read as cleanly compliant.
That is worth noticing rather than smoothing over. The two instruments ask
different questions. A judge is shown one patch and asked whether that patch takes
the ruled-out path, which a clear case answers even when the rule is vague at its
edges. An extractor is asked to write down a boundary that would settle any
implementation, which is the harder question and fails on vagueness the judge
never had to meet.
So the sets are not expected to coincide, and neither is a check on the other.
Record-Id: r-v7specseight
Provenance: authored
Certainty: firm
Blast: local
Undo: easy
Ruled-out: treating a clean control judgement as evidence the boundary is drawable | the judge saw one implementation, and a rule can classify that one and still leave the boundary unwritten
Limit: eight of seventeen from one extractor, and no comparison exists yet
Limit: zero lexical choices is a property of these eight rules and their repositories, not evidence the priority ladder is being honoured -- that is what the red-team is for
Verified: eight specifications present, each naming a detection strategy with a reason the rungs above it cannot decide the case, alternate forms, and both risk directions
… they fail the same way
Ten of seventeen from the first extractor, five reporting the boundary as
unresolvable. The reasons are not five different problems:
literally does not say whether a frozen count, hash or snapshot counts
a badge undefined, and the tree carries CI and licence badges the scope excludes
add the two paths does not say whether classifying one of the two is already a violation
hand-maintained undefined, and provenance is not recoverable from a finished tree
prose field along with derives, fixes numerically, and literal digest
Every one is a rule that uses a term it never defines, where the recorded reason
reaches further than the recorded words. These decisions were written by people
for people, in a commit trailer, and they read perfectly well that way. Asking
them to serve as machine-decidable predicates over a finished tree is a harder
demand than they were written to meet.
The fourth is a different and worse case. Hand-maintained is about how a file
came to exist, and a finished tree does not record that. The oracle's input
boundary is the final tree and the spec, so this is not a rule that is vague at
its edges -- it is a rule whose predicate no admissible oracle can evaluate at
all. Vagueness might be resolved by a careful reading. This cannot.
None of this decides anything yet. The second extractor has not run, the
registered gate fires on disagreement between the two, and a candidate the first
could not draw may still be drawable. It is committed now because the pattern was
visible at five and stating it after the second extractor agrees would read as
having been assembled to fit.
Record-Id: r-v7specsunresolvable
Provenance: authored
Certainty: firm
Blast: system
Undo: easy
Ruled-out: relaxing the final-tree input boundary so a provenance predicate becomes decidable | the boundary exists so an oracle cannot see the arm, and widening it to rescue one candidate reopens exactly what it was written to close
Ruled-out: rereading the five rules more generously until a boundary appears | the extractor was asked for a boundary and reported the gap instead, which the prompt names as the right answer; overriding it is choosing the reading that keeps the study alive
Ruled-out: reporting the rate before the second extractor as a result | it is an observation from one instrument, and the preregistration puts the gate on two
Limit: ten of seventeen, one extractor, no comparison. The final rate may differ in either direction
Limit: this says these decisions are hard to machine-decide from a final tree. It does not say they are bad decisions, and every one of them reads clearly to a human
Verified: five unresolvable specifications each name the specific undefined term and why the recorded reason and the recorded words point in different directions; none declines without a reason
…hat cannot be
Seventeen specifications from ORACLE-SPEC-A, each written from the rule and the
repository at the frozen snapshot and nothing else -- no control, no
implementation, no arm, no outcome, and not the other extractor, which has not
run.
drew a boundary 12
reported it unresolvable 5
chose a lexical instrument 0
alternate forms listed median 7, range 5-8
Zero lexical choices is the number worth keeping. The one earlier oracle in this
repository, written in v5, is a thirty-line regex over six fixed paths, and the
priority ladder exists because that is what an oracle becomes when nobody asks it
to justify its instrument. Every one of these twelve argued its way down the
ladder instead: one rejects black-box outright because a concrete store and a
port-backed store produce identical artifacts, and lists renaming, callbacks and
dependency-injection containers among the forms a check has to survive.
The five that could not draw a boundary fail the same way, and the pattern is in
the previous commit: a term the rule never defines, with the recorded reason
reaching further than the recorded words. One of the five is worse than vague --
hand-maintained is a claim about how a file came to exist, and the oracle's
input boundary is the finished tree, which does not record that.
Record-Id: r-v7specadone
Provenance: authored
Certainty: firm
Blast: local
Undo: easy
Ruled-out: starting the second pass before this one finished | two extractors that must agree are only independent if neither can see the other, and the machine now runs one heavy job at a time by owner instruction
Ruled-out: reading five of seventeen as the study's answer | one extractor is not the registered gate, and treating it as one would be deciding on half the evidence because the half arrived first
Limit: one extractor. The comparison does not exist yet and the final unresolvable rate may move in either direction
Limit: zero lexical choices describes these seventeen rules in these two repositories. It is not evidence the ladder is being honoured, which is what the oracle red-team is for
Limit: the lifecycle conflict one specification reports -- later history adopting the ruled-out approach without a supersession -- is that extractor's reading of the tree and has not been checked against the record the decision came from
Verified: 17 specifications present, each naming a detection strategy with a reason the rungs above it cannot decide the case, alternate forms, and both risk directions; the five unresolvable ones each name the specific undefined term rather than declining; counts and per-candidate summary written to preflight/oracle-spec-a.json
Five specifications into the second pass, and the two extractors disagree about
which rules can be pinned down at all -- but not in a way that makes one of them
the strict reader.
v4-002ffd1e428c572a A could not draw it, B did
v4-8f24735524874167 A drew it, B could not
Neither extractor is consistently the tighter one. Which rules read as vague
changes with who is reading, which is a property of the rules rather than a bias
in either session.
Where B drew the boundary A could not, it also reports the lifecycle as active
with no supersession, while A reported that later repository history adopts the
ruled-out approach without recording one. Same tree, opposite reading of whether
the rule is still in force. That is a larger disagreement than resolvability and
is exactly what the registered third adjudication exists for.
Comparing the unresolvable flags is only a preliminary signal. The registered
question is whether two specifications would ever classify the same tree
differently, and two that both drew a boundary can still draw different ones.
That comparison runs on all seventeen once this pass finishes.
Record-Id: r-v7specbsplit
Provenance: authored
Certainty: firm
Blast: local
Undo: easy
Ruled-out: reading the extractor that drew a boundary as the better one | a boundary drawn where the rule does not settle it is a reading someone chose, and the whole reason for asking twice is that nobody should choose it silently
Ruled-out: running the third adjudication for these two now | the machine runs one heavy job at a time by owner instruction, and the second pass is still going
Limit: five of seventeen from the second pass. The direction and rate of splits may change
Limit: agreement on the unresolvable flag is not agreement on the boundary, and this commit compares only the flag
Verified: five specB specifications present alongside their specA counterparts; both split candidates confirmed by reading each pair's unresolvable field and, for the first, its lifecycle field
both drew a boundary 3
split 3
neither could draw one 3
The three neither could draw are the ones to watch. A split is a disagreement and
the preregistration sends it to a third reading. Two independent sessions both
reporting that a rule's own words do not settle where its boundary falls is not a
disagreement -- it is agreement about the thing the study needs and cannot get.
That is not the registered gate firing yet. The gate compares boundaries, not
flags, and the comparison has not run; a third reading may still draw what
neither of the first two would. Recorded now because the rate is what it is at
nine, and a rate quoted after the comparison would be quoted knowing what it
implies.
Record-Id: r-v7specbnine
Provenance: authored
Certainty: firm
Blast: local
Undo: easy
Ruled-out: treating two unresolvable reports as a finished verdict | the comparator and the third reading are both registered steps and skipping them because the flags look decisive is deciding before measuring
Ruled-out: quoting only the split count and leaving the both-unresolvable count for later | the second number is the one that matters and holding it back until the comparison would be choosing when to say it
Limit: nine of seventeen. Three of nine is a rate on half a pass
Limit: this compares a boolean each extractor set. Two specifications can agree that a boundary exists and still draw different ones, which no count here would show
Verified: nine specB specifications present with their specA counterparts; the three-way split computed from each pair's unresolvable field
… the same four rules
both drew a boundary 6
split 3
neither could draw one 4
Four of the five the first extractor could not pin down came back unresolvable
from the second as well, and the two sessions never saw each other. The one that
moved is v4-002ffd1e428c572a, where the second drew a boundary the first
declined -- and reported the rule's lifecycle as active where the first found
later history adopting the ruled-out approach with no supersession.
Two independent readers reaching the same verdict on the same four rules is
different in kind from one reader's report. It does not make the verdict correct,
but it removes the easiest explanation for it, which is that one session was
being unusually strict.
Record-Id: r-v7specbthirteen
Provenance: authored
Certainty: firm
Blast: local
Undo: easy
Ruled-out: calling the four settled and stopping the pass | four of seventeen is not the study's answer and the remaining four candidates are still running
Limit: thirteen of seventeen, and the comparison of boundaries has not run at all. Agreement on a flag is not agreement on where a boundary falls
Limit: two sessions of the same model family. Independent of each other, not independent of whatever that family finds hard to pin down
Verified: thirteen specB specifications present with their specA counterparts; the four repeated unresolvable candidates identified by name from each pair's field
Thirty-four sessions, seventeen candidates, two independent readings each. Each
saw the rule and the repository at the frozen snapshot and nothing else -- no
control, no implementation, no arm, no outcome, and not the other reader. The
passes ran end to end rather than paired per candidate, so no session could warm
on what the previous one had concluded.
both drew a boundary 9
one drew it and the other could not 4
neither could draw one 4
The four neither could draw are the study's problem, not its noise:
v4-377f04276465b59d the rule bans "a badge" and never says which badges
v4-9b42b1951da730e1 "add the two paths" does not settle the one-path case
v4-dd4a74ba2b628991 "hand-maintained" is provenance, which a finished tree does not record
v4-e7587b2b65750306 "prose field", "derives", "fixes numerically", "literal digest", none defined
Two readers who never saw each other reached that verdict on the same four rules.
That does not make the verdict right, but it removes the cheapest explanation --
that one session happened to be strict.
The four splits go to a third reading, which is what the preregistration
registers for a disagreement. The four neither drew are not a disagreement, and
the preregistration's answer for an unresolved semantic boundary is to end the
study rather than reduce the population. That is not being invoked here: the gate
compares boundaries and this commit compares a boolean, and the comparison step
has not run.
Record-Id: r-v7specpasses
Provenance: authored
Certainty: firm
Blast: system
Undo: easy
Ruled-out: running a third reading on the four neither drew | the registered third reading resolves a disagreement between two, and these two agree; using it to find a boundary they both refused would be asking until the answer changes
Ruled-out: pairing the two extractions per candidate to finish sooner | the second reader would run right after the first on the same rule, and independence is the only property that makes two readings worth more than one
Ruled-out: reporting these counts as the gate | the gate is whether two specifications would classify some tree differently, which no flag can answer
Limit: two sessions of one model family. Independent of each other, not independent of what that family finds hard to pin down, and a different family might draw all four
Limit: this compares flags. Nine candidates whose readers both drew a boundary may still have drawn boundaries that disagree, and that is the next measurement
Verified: 34 specifications present, one pair per candidate; classification computed from each pair's unresolvable field; the four repeated cases identified by name with the undefined term each names
…have passed
Four of nine candidates whose two readers both drew a boundary have now been
compared on the boundaries themselves, by asking a third session to build a
concrete tree the two would classify differently.
agree 1
disagree 3
Both outcomes matter and neither is visible from the unresolvable flags, which
were false on all four.
v4-77e1745655a235ce agrees after seven failed attempts to split it -- a
hard-coded frozenset of the current kinds, a single callable registry, a separate
enum consulted independently, an alternate factory that bypasses validation. Both
specs classify every one of them the same way. The boundary they share is
operational: an evidence-kind vocabulary is acceptable only when it derives from
the same active producer registration that read_evidence dispatches.
v4-34aef026d81c2f6b disagrees, and the axis is scope. One spec reads the rule
across the whole workflow-validator-projection pipeline and counts a
discovered-and-pruned input that reaches operational readiness. The other confines
the rule to execution-view renderer source selection, which permits exactly that
path. The separating case is a specific edit to scripts/validate-planning.mjs,
written precisely enough to apply.
The attempt counts are doing useful work on their own. Agreements arrive with
seven attempts behind them and disagreements with three, because a comparator
that finds a separating tree stops there and one that cannot keeps trying. An
agreement with an empty attempt list would be worth nothing, and the prompt says
so.
Record-Id: r-v7agreefour
Provenance: authored
Certainty: firm
Blast: local
Undo: easy
Ruled-out: comparing the two specifications by how similarly they read | two specs can use nearly the same words and split on a real case, and two can describe one boundary in unrelated language; only classification behaviour settles it
Ruled-out: treating both readers drawing a boundary as agreement | three of these four drew different boundaries, and a flag comparison would have passed all four
Limit: four of nine, and the five remaining may move the rate in either direction
Limit: one comparator session per candidate. A separating case that exists and was not constructed reads here as agreement, so agreement is bounded by how hard one session tried
Verified: each comparison recorded with its attempted refutations, its separating case where one exists, and which anonymised spec calls that case a violation; the anonymisation ordering is recorded per candidate and fixed by a hash of the candidate id
… four have nowhere to go
The nine candidates whose two readers both drew a boundary have been compared on
the boundaries themselves. A third session saw the two specifications anonymised,
with the ordering fixed by a hash of the candidate id, and was asked to build a
concrete tree the two would classify differently.
agree 3 attempts before giving up: 5, 6, 7
disagree 6 attempts before succeeding: 0, 2, 3, 3, 3, 3
The attempt counts separate completely. Every agreement survived at least five
constructed attacks; every disagreement fell to at most three. Nothing sits in
between, which is what one would expect if the comparator behaves consistently:
it stops when it finds a separating tree and keeps going when it cannot.
One disagreement needed no attempt at all. Two readers of the same rule about
protected test coverage answered its central question in opposite directions --
whether deliberately narrowing a focused test lane and remeasuring the expected
total is permitted. That is not an edge case anyone had to hunt for.
Where the seventeen now stand:
boundary settled 3
goes to a third reading 10
neither reader drew a boundary 4
The ten are the four where one reader drew a boundary the other declined, plus
the six where both drew and drew differently. The preregistration registers a
third reading for exactly this.
The four are not in that queue and will not be put there. Two independent readers
agreed the rule's own words do not settle where its boundary falls. A third
reading is registered to resolve a disagreement, and there is none to resolve;
asking a third session to find what two refused would be asking until the answer
changes.
Record-Id: r-v7agreecomplete
Provenance: authored
Certainty: firm
Blast: system
Undo: easy
Ruled-out: sending the four to a third reading anyway | the registered tie-break resolves disagreement between two readers, and using it on their agreement converts a gate into a retry
Ruled-out: counting the six disagreements as near-agreement because some took three attempts | a separating tree either exists or does not, and how long it took to build says something about the comparator rather than about the boundary
Ruled-out: treating three settled as a result | this is a count of boundaries, not of oracles, and nothing downstream has been built or validated yet
Limit: one comparator session per candidate. A separating case that exists and was not constructed reads as agreement here, so the three settled are bounded by how hard one session tried
Limit: two extraction sessions of one model family, plus one comparator from the same family. Independent of each other, not of what the family finds hard to pin down
Limit: three settled is where this stands before the third readings, not a final count. It can only go up
Verified: nine comparisons recorded with attempted refutations, separating case where one exists, and which anonymised spec calls it a violation; anonymisation ordering recorded per candidate; the full seventeen-candidate classification and the third-reading queue written to preflight/spec-agreement.json
…recorded scope
Ten candidates went to a third reading: four where one of the first two drew a
boundary the other declined, six where both drew and drew differently. The reader
gets the rule, the repository, and both earlier attempts anonymised, and is asked
the original question rather than which attempt to prefer. Picking would make it a
vote, and the preregistration registers a third reading.
Three in:
v4-002ffd1e428c572a rule does not settle it
v4-0ecd7426eebc1cab boundary written
v4-34aef026d81c2f6b boundary written
The two it resolved were resolved on evidence rather than preference. For
v4-0ecd7426eebc1cab the earlier readers split on whether the rule reaches
equivalent seams beyond gitseed/ports.py, and the third reading answered from the
recorded scope, which names that file and nothing else: the wider reading reads
past what was written down. Inside that file the repository's own RunPorts
structure and the rule's replay rationale identify the prohibited thing exactly.
The one it did not resolve, it declined for a stated reason. The split turns on
whether "literally" prohibits an explicit path enumeration or any fixed exhaustive
census -- a digest, a snapshot, a generated constant. The recorded scope settles a
second question the readers also split on, and settles nothing about this one. The
rule's stated reason favours the broad reading and its operative words say
"literally", and neither says whether the boundary is the representation or the
per-ticket edit it forces.
That is the shape worth keeping: a third reading that resolves where the record
contains an answer and reports where it does not, rather than producing a verdict
either way because one was asked for.
Record-Id: r-v7thirdfirst
Provenance: authored
Certainty: firm
Blast: local
Undo: easy
Ruled-out: asking the third reader to choose between the two earlier attempts | that is a majority vote, and two readers disagreeing has already shown the rule does not settle itself by being read again more confidently
Ruled-out: showing the third reader which attempt came from which extractor | the ordering is fixed by a hash of the candidate id so it cannot be chosen after the fact, and provenance would give it something to defer to
Limit: three of ten. The rate at which third readings resolve is not established by three
Limit: a third reading is one more session of the same model family. It is independent of the first two and not of what that family finds hard to pin down
Verified: each third reading recorded with the split it identified, the side the rule's words support where it resolved, and the specific undefined term where it did not; anonymisation ordering recorded per candidate
…m could apply
Every one of the seventeen was read twice, independently, by sessions that saw
the rule and the repository at the frozen snapshot and nothing else. Where both
drew a boundary, a third session tried to build a tree they would classify
differently. Where they split, a fourth read the rule again with both attempts in
front of it, anonymised, and was asked the original question rather than which
attempt to prefer.
both readers drew the same boundary 3
third reading resolved the split 5
------------------------------------------------
boundary settled 8
third reading found the rule does not settle it 5
both readers agreed it cannot be drawn 4
------------------------------------------------
not settled 9
The third reading split its ten exactly five and five, and it resolved on
evidence rather than preference. Where it wrote a boundary it pointed at
something in the record: for two candidates the recorded scope names one path,
so a reading that reaches equivalent seams elsewhere reads past what was written.
Where it declined it named the term -- whether "literally" prohibits an explicit
path list or any fixed exhaustive census, whether narrowing a focused test lane
and remeasuring its expected total is the thing the rule forbids.
The most contested candidate came out settled. v4-f3c960a48273132c split both its
control judges, drew a VIOLATION_CONFIRMED on a builder that was never told the
decision, and had the one imported Bad A patch that could not be replayed. Its
boundary is written: the rule's subject is the structural declaration of scoring
and screening ports inside the one recorded path, not whether a replacement seam
is reachable in production. That also explains its judges -- one was looking at
whether the port was declared and the other at whether it was used.
Those are different questions and they come apart. A judge asks whether one patch
took the ruled-out path, which a clear case answers even where the rule is vague.
A specification has to settle every implementation. This candidate is vague for
the first and settled for the second, and elsewhere in this corpus it runs the
other way.
What Phase 5 does not decide: whether the eight settled boundaries can be
implemented as deterministic oracles that survive a red-team, and what the nine
unsettled ones mean for a study whose population may not shrink. Both are next.
Record-Id: r-v7phase5close
Provenance: authored
Certainty: firm
Blast: system
Undo: easy
Ruled-out: sending the four both-readers-declined candidates to a third reading | the registered third reading resolves disagreement, those two agreed, and asking a third for what two refused is asking until the answer changes
Ruled-out: letting the third reader pick between the earlier attempts | that is a majority vote of three sessions, and two readers disagreeing has already shown the rule is not settled by being read again with more confidence
Ruled-out: counting the five third-reading declines as softer than the four agreed declines | both mean no boundary was written, and grading them by how many sessions it took would be reading the process rather than the result
Limit: every session in this phase is the same model family. Independent of each other, not independent of what that family finds hard to pin down, and a different family might settle more or fewer
Limit: agreement is bounded by one comparator session's effort. A separating tree that exists and was not constructed reads here as agreement, so three of the eight rest on a failure to refute
Limit: eight settled is a count of boundaries, not of working oracles. Nothing has been implemented, run against a control, or attacked
Verified: 17 candidates each with two independent specifications; 9 boundary comparisons with attempted refutations and separating cases recorded; 10 third readings with the split each identified and either the side the record supports or the specific undefined term; all ordering anonymisations fixed by candidate-id hash and recorded; counts written to preflight/phase5-summary.json
… and nine are not
CDEB-Fresh v7 reached TERMINAL_HOLD_FINAL before any product-effect episode.
Eight of the fixed 17 decisions yielded a semantic boundary precise enough for
deterministic oracle construction and nine did not. Because the preregistered
population was fixed at all 17 tasks and unresolved ambiguity was terminal under
v7, the population was not reduced post hoc. This result concerns deterministic
machine adjudicability, not the causal effect of CommitLore delivery.
fixed benchmark population 17
semantic boundary settled 8
semantic boundary unresolved 9
measured product-effect episodes 0
Running the eight would have been a study of the decisions that happen to be
machine-adjudicable, reported as though it were a study of the seventeen. The
floor exists to stop that and it was not moved.
The nine fail the same way: a rule turning on a term it never defines, with the
recorded reason reaching further than the recorded words -- "literally", "a
badge", "add the two paths", "prose field". One is not vagueness at all.
"Hand-maintained" is a claim about how a file came to exist, and the oracle's
admissible input is the finished tree, which does not record that. No rereading
fixes an instrument that cannot see the property.
These decisions were written by people for people in a commit trailer and read
perfectly well that way. What v7 measured is how far that survives being turned
into a machine predicate.
Also in this commit: the v6 test asserting `last_terminal_study_id` is
"cdeb-fresh-v6". That field names whichever study ended most recently and moved
the moment this one terminalised. I had already ruled this exact shape out in
r-v7opens -- "a test that has to be edited on every transition is not recording a
durable fact" -- and then pinned a different moving field in the same test while
fixing the first one. What holds for v6 whatever follows is that it ended and can
never be named active again, and that is what it asserts now.
Record-Id: r-v7terminal
Provenance: authored
Certainty: firm
Blast: system
Undo: easy
Ruled-out: running v7 on the eight settled candidates | the preregistration fixed the population at seventeen before any task existed, and reducing it after seeing which ones resolved is the discretion the floor was written to remove
Ruled-out: relaxing the oracle input boundary so a provenance predicate becomes decidable | that boundary exists so an oracle cannot see the arm, and widening it to rescue one candidate reopens what it was written to close
Ruled-out: reporting eight of seventeen as a result about CommitLore | no episode ran, no arm was assigned, and nothing here bears on whether automatic decision delivery helps an agent
Limit: every session in the boundary work is one model family. Independent of each other, not of what that family finds hard to pin down, and a different family might settle more or fewer than eight
Limit: three of the eight rest on one comparator session failing to construct a separating tree, which is weaker than a proof that none exists
Limit: eight settled counts boundaries, not oracles. None was implemented, run against a control, or attacked
Limit: the rebuilt Good controls are v7 artifacts occupying v6 slots, and four of the thirty-four were not read as cleanly compliant by their blind judges
Verified: RESULT.md carries the registered terminal sentence verbatim; STATUS is TERMINAL_HOLD_FINAL with measured_run_allowed false and product_effect_rows 0; ACTIVE-STUDY is no-active-study with last_terminal_study_id cdeb-fresh-v7; the transition row hashes the phase-5 summary it rests on; guard-mutations exits 0 with 67 bound; tsc clean on both tsconfigs; vitest 3747 passed, 13 skipped, 0 failed; the rewritten v6 assertion observed failing when the declaration names v6 active and passing when restored
CommitLore — record lintTrailers: clean — 27 commits in Active constraints for the paths this PR touchesLimits (70)
Ruled out (86)
Trailer violations fail this check. Active constraints are informational — they are what the repository already decided, not a verdict on this PR. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
How a boundary was settled
Each decision was read twice by independent sessions that saw the rule and the
repository at the frozen snapshot and nothing else — no control, no
implementation, no arm, no outcome, and not each other. Where both drew a
boundary, a third session tried to build a tree they would classify differently;
agreement is what survived that. Where they split, a fourth read the rule again
with both attempts anonymised and was asked the original question rather than
which attempt to prefer.
The comparator's attempt counts separate cleanly: every agreement survived at
least five constructed attacks, every disagreement fell to at most three.
Why the nine
They fail the same way — a rule turning on a term it never defines, with the
recorded reason reaching further than the recorded words:
One is not vagueness. "Hand-maintained" is about how a file came to exist, and
the oracle's admissible input is the finished tree. The rule settles its own
question and no admissible instrument can apply it.
These decisions were written by people for people in a commit trailer, and they
read perfectly well that way.
Two instrument defects repaired in flight
diff. Bad A patches survived only because the blind judges had been handed a
diff; Good A and Good B did not survive at all. 34 compliant controls were
rebuilt, all passing both acceptances, and their patches committed. (
v7-d003)git diff, which omits files the buildercreated. All 17 imported patches carry zero new-file entries, and the one Bad A
that created a module could not be replayed. Rebuilt, and confirmed a violation
by two blind judges. (
v7-d004)Both were corrected while the study held zero measured rows.
What this is not
No episode ran and no arm was assigned. Nothing here supports or refutes any
claim about whether automatic decision delivery helps an agent.
Verification
Every session in the boundary work is one model family — independent of each
other, not of what that family finds hard to pin down. Three of the eight rest on
one comparator failing to construct a separating tree rather than a proof none
exists. Eight settled counts boundaries, not working oracles: none was
implemented, run against a control, or attacked.