chore(do not merge): Experiment extending to rpc tests via hive - #3340
Draft
kevaundray wants to merge 100 commits into
Draft
chore(do not merge): Experiment extending to rpc tests via hive #3340kevaundray wants to merge 100 commits into
kevaundray wants to merge 100 commits into
Conversation
Adds `openrpc.json` from execution-apis `v1.0.0-beta.7` as the authority for JSON-RPC response shapes. Upstream gitignores this file — it is a build artifact of `src/*.yaml` produced by their Go `specgen` tool, so it is available neither from a clone nor as a release asset. Generated with the same flags hive's `rpc-compat` simulator uses, so the schema validated against here matches the one clients are tested against upstream. `-deref` is kept for that reason; skipping it saves only 0.5MB. `just refresh-openrpc [tag]` regenerates the file and is the only supported way to update it, keeping refreshes a clean diff. Also ignores `docs/plans/`, for local design notes that are not part of the published mkdocs site.
…ures Adds the projection layer that maps filled fixture data onto the JSON-RPC response objects the OpenRPC schema describes, covering `eth_getBlockByNumber` (hash form) and `eth_getTransactionReceipt`. Expected values come from the Python spec's own output rather than a recorded client response, so no client is the reference and vectors can exist before any client implements a feature. Everything here is a pure function of data `t8n` already returned; no execution is repeated. Three differences from the consensus types drive the design: - Quantities must use minimal hex. The schema's `uint` pattern rejects the `ZeroPaddedHexNumber` form fixtures store (`0x01`), so the projection converts. - Absent and null are distinct. `status`, `root` and `blobGasUsed` are omitted when inapplicable, while `to` and `contractAddress` are always present and merely null; `model_dump(exclude_none=True)` cannot express that split, so `conditional_fields` drives it explicitly. - `logIndex` is block-scoped rather than transaction-scoped, so receipts are projected together. Verified by projecting 15,624 blocks and 18,470 receipts from filled Berlin through Amsterdam fixtures with zero schema violations, with a negative control confirming the validator rejects zero-padded quantities, missing required fields, wrong types and malformed addresses. The full-object form of a block, returned when a request sets `fullTransactions`, needs a transaction projection and is not implemented.
…rpc` Wires the projection into filling. A test marked `rpc` gains an `rpc` section in its blockchain fixture holding the JSON-RPC calls its chain implies, together with the response the spec requires. Nothing about the calls is author-supplied. They are read off the filled chain: each block contributes a lookup by number and by hash, each transaction a receipt lookup. The marker is only a switch, which is why it carries no parameter information. Emission is opt-in, following `verify_sync` and `BlockchainEngineSyncFixture`. The derivation branches on the shape of the blocks rather than on what a test exercises, so a `PUSH0` test and an `MCOPY` test drive identical code paths; emitting corpus-wide would grow the release artifact measurably without adding coverage. Also adds schema validation against the vendored OpenRPC document, which is the phase-1 comparison mode. Note the schema is fork-agnostic and cannot express "required at Cancun", so it is a floor rather than a ceiling — exact comparison against a fork-parameterized expectation is what ultimately pins those fields. The Amsterdam test covers EIP-7708, where plain value transfers emit logs and so a receipt carries entries for transactions touching no contract. Three transfers in one block exercise block-scoped `logIndex` numbering, which a per-transaction counter would get wrong. Verified by filling: the marked test emits 5 schema-conformant calls with `logIndex` running 0,1,2 across the block, and the 77 unmarked cases in the same directory emit no section at all.
Replays a fixture's stored `rpc` section against a live client and reports every mismatch together. Fixtures without the section, which is all but the tests marked `rpc`, are skipped without a round trip. `consume rlp` is the right and only home for this today: it is the sole hive simulator consuming `BlockchainFixture`, which is where the `rpc` section lives. `consume engine` reads `BlockchainEngineFixture`, and `consume direct` drives an offline binary with no RPC server at all. That also matches the chain-import route the design settled on — RLP import plus a forkchoice update. Comparison is schema-based for now, catching missing required fields, wrong types and malformed values, but not a wrong value of the right shape. The stored results are what exact comparison will use later. Error responses compare on code only. Their wording is client-specific and unspecified, so matching on it would fail conforming clients — the same conclusion hive's `rpc-compat` reached with `redactErrorMessages`. Reporting needed care. Most `eth_` results are a `oneOf` of an object and null, so one bad field surfaces as "is not valid under any of the given schemas" against the whole response, naming nothing and dumping the entire object. Generic relevance heuristics then prefer the null branch, because "is not of type 'null'" is the shallower complaint. Dropping root-level type mismatches before descending recovers the field that actually broke, turning the failure into `number: '0x01' does not match '^0x(0|[1-9a-f] [0-9a-f]*)$'`. Verified with 10 tests covering a conforming client, missing fields, zero-padded quantities, unexpected and expected errors, message wording being ignored, and batched reporting of every failure at once.
Fixtures store the wire method name (`eth_getBlockByNumber`) because that is what the OpenRPC schema keys on and what a client team reads in a failure. The RPC clients are namespaced and prepend their own prefix, so passing the stored name through produced `eth_eth_getBlockByNumber` and every call came back -32601. Mocks hid this completely: a `MagicMock` accepts any method name, so the unit tests passed while nothing worked against a real node. Found by running `consume rlp` against go-ethereum under hive. A method outside the client's namespace now raises rather than being mangled, so `debug_getRawBlock` through an `eth` client names our mistake instead of reporting a spurious -32601 against the client.
Nothing validated the stored expectation. `FixtureRPCCall.result` is `Any` and `method` is a bare `str`, so a fixture could carry `"result": "not a block at all"` for a method that does not exist and both the model and `checkfixtures` accepted it — verified against plain strings, JSON-encoded strings, numbers and null. That failure mode blames the wrong party. A garbage expectation reaches a client team as "your eth_getTransactionReceipt response is wrong", sending them to debug an assertion that was never satisfiable, and the mistake is in a released artifact by then. Validating at the point of derivation keeps the bad state out of the artifact rather than detecting it later. `ProjectionError` is deliberately distinct from `SchemaViolationError`: one says the projection is broken, the other says a client is, and conflating them is what makes this class of bug expensive. Unknown method names fall out of the same check, since the schema has no result definition to validate against. Verified by mutation: reintroducing the zero-padded-quantity regression makes `fill` fail with "derived expectation for eth_getBlockByNumber is not schema-conformant ... This is a projection bug, not a client bug", where previously it emitted a fixture no client could ever pass.
Phase 2. Schema validation alone cannot catch a wrong value of the right
shape, which is the entire reason the expectation is stored rather than
recomputed by the consumer. Comparison now runs as two layers: the schema
first, because it gives a precise message for a missing field or malformed
value, then a field-by-field comparison reporting each difference with its
path, e.g. `cumulativeGasUsed: expected '0x1234', got '0xac5c'`.
Two relaxations are required for correctness, not convenience:
- Hex case is normalized. The schema's address pattern is
`^0x[0-9a-fA-F]{40}$`, so a client returning EIP-55 checksummed
addresses conforms and byte-wise comparison would wrongly fail it.
- The walk follows the expectation, not the response, so a field we do not
model is not treated as a client error. The projection knowingly omits
`withdrawals`, which go-ethereum returns on every post-Shanghai block,
and the block schema permits additional properties. Receipts are
stricter — that schema sets `additionalProperties: false` — and the
schema layer draws that line rather than the value layer.
Differences are capped per call so one badly wrong response cannot bury
the rest of the run.
Verified against go-ethereum v1.17.6 under hive: five expectations match
value for value, and corrupting a stored `cumulativeGasUsed` to another
well-formed quantity fails with the path-anchored diff while passing
schema validation, confirming the value layer is doing the work.
kevaundray
commented
Aug 8, 2026
| REFERENCE_SPEC_GIT_PATH = ref_spec_7708.git_path | ||
| REFERENCE_SPEC_VERSION = ref_spec_7708.version | ||
|
|
||
| pytestmark = [pytest.mark.valid_from("EIP7708"), pytest.mark.rpc] |
Contributor
Author
There was a problem hiding this comment.
pytest.mark.rpc allows us to convert existing tests in to rpc test
kevaundray
commented
Aug 8, 2026
Comment on lines
+103
to
+105
|
|
||
| # Local design docs; not part of the published mkdocs site | ||
| docs/plans/ |
Contributor
Author
There was a problem hiding this comment.
Planned this initial version out with AI to get a prototype
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## forks/amsterdam #3340 +/- ##
===================================================
- Coverage 93.53% 93.27% -0.27%
===================================================
Files 624 624
Lines 37070 37355 +285
Branches 3394 3424 +30
===================================================
+ Hits 34675 34844 +169
- Misses 1645 1715 +70
- Partials 750 796 +46
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
kevaundray
commented
Aug 8, 2026
Comment on lines
+401
to
+420
| [group('housekeeping')] | ||
| refresh-openrpc tag="v1.0.0-beta.7": | ||
| #!/usr/bin/env bash | ||
| set -euo pipefail | ||
| # Flags mirror hive's rpc-compat Dockerfile, so the schema we validate | ||
| # against matches the one clients are tested against upstream. | ||
| dest="$(pwd)/packages/testing/src/execution_testing/rpc/schemas/openrpc.json" | ||
| work="$(mktemp -d)" | ||
| trap 'rm -rf "$work"' EXIT | ||
| git clone --depth 1 -b "{{ tag }}" -q \ | ||
| https://github.com/ethereum/execution-apis.git "$work" | ||
| cd "$work/tools" && go build -o specgen ./cmd/specgen | ||
| cd "$work" && ./tools/specgen -o "$dest" -deref \ | ||
| -schemas 'src/schemas' -schemas 'src/engine/openrpc/schemas' \ | ||
| -methods 'src/eth' -methods 'src/debug' -methods 'src/txpool' \ | ||
| -methods 'src/engine/openrpc/methods' -methods 'src/testing' \ | ||
| -error-groups 'src/error-groups' | ||
| echo "Regenerated from {{ tag }} ($(git -C "$work" rev-parse HEAD))." | ||
| echo "Update the pin table in the schemas README if the tag changed." | ||
|
|
Contributor
Author
There was a problem hiding this comment.
This is hacky and just a symptom of this PR vendoring in the json schema file
Contributor
Author
|
Note that this isn't completely correct: apart from the engine API methods, it seems that most methods are not versioned by fork in execution-api. There are two ways that I think would make sense for versioning:
For namespaces like EDIT: The testing machinery does it by marking fields as optional and ensuring that if we are at fork X, then all of those fields are present if they were added at fork X. (Given the discussions around REST API, might be best to leave it as is) |
go-ethereum returns `withdrawals` on every post-Shanghai block and the projection asserted nothing about it. Because unasserted fields are ignored by design, this was a silent coverage gap rather than a failure — the worst kind, since the suite looked like it was checking the block. Empty and absent are distinct and both are meaningful. From Shanghai a block reports `[]` when it has no withdrawals; an earlier block has no such field at all. `withdrawals` is therefore a conditional field, and the empty list survives serialization rather than being dropped with the `None`s. Quantities need the usual conversion: fixtures store withdrawal `index`, `validatorIndex` and `amount` zero-padded, and the schema requires minimal hex. Verified against go-ethereum v1.17.6 under hive with two withdrawals carrying distinguishable values, matching field for field. Corrupting one amount fails with `withdrawals/1/amount: expected '0x99', got '0x0'`, confirming the value layer reaches into nested arrays.
`consume rlp` and `consume engine` exercise deliberately different client code paths — bulk import at start-up versus `engine_newPayload` — and receipt storage and log indexing are plausibly where an import-path specific bug would live. Asserting the same expectations on both is what makes that difference visible. This needed more than calling the helper from `test_via_engine`. A `FixtureExecutionPayload` carries `receiptsRoot` but no receipts, and its transactions are raw RLP with no decoded senders, so the projection cannot run against a finished engine fixture at all. The blocks have to be assembled during generation, where the transition tool output is still in hand, which means the decision to emit must be known before filling rather than applied to the finished fixture. `BlockchainTest.emit_rpc_expectations` therefore carries that decision, set from the marker by the filler before `generate`, replacing the post-hoc hook. Both formats now derive through one code path, and blocks are only assembled in the engine path when they will actually be used, so unmarked tests pay nothing. Verified against go-ethereum v1.17.6 under hive: both simulators replay four expectations and pass, and corrupting a stored `logIndex` in the engine fixture fails with `logs/0/logIndex: expected '0x7', got '0x0'`.
Takes the derived surface from two methods to eleven: both forms of
`eth_getBlockBy{Number,Hash}`, `eth_getBlockTransactionCountBy{Number,Hash}`,
`eth_getBlockReceipts`, `eth_getTransactionByHash`,
`eth_getTransactionByBlock{Number,Hash}AndIndex`, `eth_getLogs` and
`eth_blockNumber`, alongside the existing receipt lookup.
The transaction projection is the substantive piece, and the schema splits
per type in ways that are easy to get wrong: legacy carries `v` while
typed transactions carry `yParity`; 1559 and later require `gasPrice` as
well as the fee caps, holding the effective price the sender never set;
and 4844 and 7702 cannot be creations, so `to` is required rather than
nullable there.
Two things the schema forced:
- `RPCBlock.transactions` is untyped. Pydantic resolves
`List[Hash] | List[Dict]` by trying `Hash` on each entry first, and
`Hash` raises `TypeError` rather than a validation error, so the union
never falls through to the object form. Hashes are stringified in the
projection for the same reason — with `Any` there is no `Hash`
serializer and the raw bytes fail a utf-8 decode.
- `eth_getLogs` is emitted only when the chain produced a log. Its result
is `oneOf` an array of log objects and an array of hashes, and an empty
array satisfies both, which `oneOf` forbids — so a legal empty response
is unrepresentable. That looks like an upstream schema bug.
Verified against go-ethereum v1.17.6 under hive: 25 expectations over a
block mixing legacy, 2930, 1559 and a creation, with logs and a
withdrawal, replayed and matched through both `consume rlp` and
`consume engine`.
Adds `eth_getBalance`, `eth_getTransactionCount`, `eth_getCode` and `eth_getStorageAt` at the head block, taking the derived surface to fifteen methods. Accounts are narrowed to the ones the chain actually reaches — senders, recipients, created contracts, withdrawal recipients and the fee recipient. A post-state holds every pre-allocated account, most of which a test never goes near, so asserting all of them would multiply the fixture without adding coverage. Storage is capped per account and exceeding the cap is logged rather than silently truncated. `eth_getCode` stores a digest instead of the bytecode. The client still returns the full code and is exercised exactly as before; only the stored expectation shrinks to 32 bytes. The motivation is not size alone: the code already appears in the fixture's `pre` and `postState`, so repeating it duplicates the largest field in the file to assert something the fixture already states. It also happens to remove what measurement showed was 40% of raw growth. The cost is losing a byte-level diff on failure, which for bytecode is unreadable anyway, so the mismatch reports the returned length instead. Verified against go-ethereum v1.17.6 under hive through both simulators: 51 expectations over a block mixing legacy, 2930, 1559 and a creation with logs, storage and a withdrawal. Corrupting a balance and a code digest fails with `expected '0x1234', got '0x69c84c94edf2'` and a digest mismatch reporting `0 bytes returned`.
Derivation reads its parameters off the chain, so it can only ask questions the chain answers. A reversed block range, a hash belonging to nothing, or a block tag cannot be expressed that way because no chain produces them. `RPCExpectation` lets a test declare those alongside its blocks, and they are appended to the derived section. This deliberately avoids the separate `rpc_test` format and `consume rpc` simulator the design originally called for: the section and replay machinery already exist, and `FixtureRPCCall.error_code` was added for exactly this and was until now unused. The cost is the attribution tradeoff already accepted for the derived half — a failure surfaces as a `consume` failure rather than separately. Only two outcomes are expressible: an error code, or an explicit null for a lookup that should find nothing. A hand-written *result* is rejected, because that would reintroduce the maintained expectation the whole design exists to avoid — every real value comes from the spec. Verified against go-ethereum v1.17.6: four not-found lookups emitted alongside 21 derived calls and all replayed successfully.
Derivation read parameters off the chain's own blocks, so the block tags were listed as out of reach. That is only half true: `latest` and `earliest` name nothing the chain produced, but they resolve to blocks it does have, so no test author has to supply anything for them. `earliest` is the interesting one. A fixture stores genesis as a header plus an RLP encoding rather than as a block, so nothing projected it and the one block with no parent, no transactions and a fork-dependent header was never asserted at all. `genesis_block` reassembles it; `withdrawals` is recovered from the header, since a chain committing to a withdrawals root reports an empty list at genesis while an earlier one has no such field. Verified against go-ethereum from Byzantium through Prague, which covers genesis with and without a base fee and with and without withdrawals. `safe` and `finalized` are deliberately left out, and this was measured rather than assumed. Both simulators send a forkchoice update carrying only `headBlockHash`, so go-ethereum answers `-32000 safe block not found` on both the rlp and the engine path. Asserting the head there would encode our own harness's forkchoice; asserting the error would encode its absence. `rpc-compat` avoids this by replaying a `headfcu` that sets all three, which we cannot copy — a finalized head forbids the reorgs other tests depend on. Only the full-transaction form is emitted per tag, since it contains the hash form and the hash form is already asserted by number. The head block is now tracked as a block rather than only as a number, because `latest` needs the object. Verified: 72 expectations replayed green on `consume rlp` and `consume engine` against go-ethereum v1.17.6, and a corrupted `size` on the genesis projection, a corrupted `gasUsed` on the `latest` block and a non-empty expectation for the genesis receipts each fail with a path-anchored message.
Three families of question that the design listed as hand-written turn
out to need nothing from a test author, so they are enumerated with
everything else. The alternative was extending `RPCExpectation` with a
literal result, which would have put hand-maintained values back into
test bodies — the one thing this design exists to avoid — and would have
had to be repeated in every marked test.
**Reads of an account that does not exist** are the case worth being
careful about, because they are *not* null. The state is a total
function, so an unallocated address has zero balance, zero nonce, no code
and all-zero storage. They go through the same helper as a real account
with an empty account standing in, so the values stay computed rather
than written down. The address is `keccak(head block hash)[-20:]`, which
keeps it a parameter read off the chain and beyond anything a test could
allocate; absence is still checked against the post-state first, since a
wrong absence claim would be our bug rather than a client's.
**Lookups by the zero hash** are the counterpoint: a block or transaction
that is not found really is null, and no block or transaction can hash to
zero, so the answer holds on every chain.
**Two malformed storage keys** — one nibble too long, and `0xasdf`. Both
fail while the key is decoded into a 32-byte word, before any account or
block is consulted, which makes them facts about parameter parsing rather
than about a chain. The code was measured, not guessed: go-ethereum
v1.17.6 answers `-32602 storage key too long` and `-32602 invalid hex in
storage key` on both the rlp and the engine path, and -32602 is JSON-RPC's
own code for a parameter that could not be interpreted. Only one client
was available, so a disagreement is possible; the request names an
account that exists, so a client has no other ground on which to refuse
it.
**Every state read is now asserted twice**, once naming the head block
and once naming no block at all. Omitting the parameter defaults to
latest, so the answers must agree; what it exercises is the client's
defaulting path, which the numbered form never reaches. This doubles the
cheapest section of the fixture — a state read is a short string where a
block object is a kilobyte.
Also fixes a latent gap found on the way: `account.storage or {}` drops
the storage of an account whose slots are all zero, because `Storage`
tests as false in that case. A slot the chain explicitly zeroed is
exactly where a client might wrongly report the previous value, so it is
now asserted.
Verified against go-ethereum v1.17.6 on both simulator paths, with a
negative control per behaviour: a nonzero balance for the absent account,
a nonzero word for its storage, an object where a null was expected, a
wrong value on an untagged read and `-32000` in place of `-32602` are all
reported individually and none of them passes.
An authority is the account a set-code transaction actually changes, and
nothing reached it. `touched_accounts` walked senders, recipients,
created contracts and withdrawal recipients — an authority is none of
those unless the transaction happens to be addressed to it, which is
incidental. So the two things a client must report about a delegation
were never asserted: `eth_getCode` returning the designator
`0xef0100 ‖ address` where the account previously had none, and
`eth_getTransactionCount` counting an authorization from an account that
sent nothing.
Authorities now join the touched set, read from the fixture's own
`signer` field rather than recovered again from the signature. The
delegation *target* is deliberately not added: the transaction reads its
code but does not change it, and the set is scoped to what changed.
`eth_getCode` also stops storing a digest for short code. The digest rule
exists because bytecode is large and already present in `pre` and
`postState`, but at 32 bytes or less the code is no bigger than its own
digest, so the digest costs more and says less. A delegation designator
is 23 bytes, and this is what makes its failure legible: a corrupted
expectation now reports
expected '0xef01001111...', got '0xef01006280aceac5db12798d...'
naming the account the delegation wrongly points at, instead of "digest
mismatch". Empty code likewise reads as `0x`.
The new test delegates to two different targets from one transaction
addressed to a third, unrelated account. Two targets rather than one so
the designator has to name the account it actually points at rather than
any constant; addressed elsewhere so the authorities are reachable only
through the authorization list, which is the gap this closes. A second
block calls into one authority so the delegated code runs and writes its
storage to the authority rather than the target.
Verified against go-ethereum v1.17.6 on both `consume rlp` and `consume
engine`: 67 expectations green, and corrupting the two designators and
the two authority nonces produces exactly eight individually reported
failures.
An agent worktree created under `.claude/worktrees/` is a transient checkout of this same repo. Left untracked it shows up in `git status`, gets scanned by `just static` (reporting lint errors against a sibling agent's in-progress code), and is a `git add -A` away from being committed into the tree it is a copy of.
`safe` and `finalized` are the only two block tags no chain determines. The consensus layer names them through `engine_forkchoiceUpdated` and the execution client's whole job is to remember them, so there is nothing for the spec to derive and nothing for the projection to compute. That is exactly why the tags were left out until now. Two fields make them reachable without pretending they are derived. `FixtureForkchoiceState` records the head/safe/finalized triple a consumer must declare before the tags mean anything. It lives on `BlockchainEngineFixtureCommon` alone: the Engine API is the only channel for the declaration, and `consume rlp` never opens it, so an RLP fixture carrying the field would describe an instruction its consumer cannot follow. It is also the only fixture field that tells a consumer what to *do* rather than what the chain *is*, which is worth flagging rather than smuggling in. `FixtureRPCCall.round_trip` marks an expectation whose value came from that declaration rather than from the spec. The distinction is recorded in the artifact and not merely in the code that emits it, for two reasons. A client team reading the fixture is entitled to know which assertions descend from a specification and which describe the harness — blurring them would quietly falsify the claim that every expectation here is spec-derived. And a consumer that cannot make the declaration needs something to key its skip on, which a comment in the emitter cannot provide. The flag is a plain `bool`, so `"roundTrip": false` is written on every call of a marked fixture: about 20 bytes each, 1.4 KB on a three-block test, and essentially nothing after gzip, which takes that fixture from 112 KB to 6.9 KB. A tri-state field to elide the false case would be worse code for a compressed saving of roughly zero.
Given a declaration, the two tags become assertable: each names a block of this chain, and that block's projection is already computed for its own number and hash. `derive_rpc_calls_for_blocks` takes the tags as an optional mapping and emits `eth_getBlockByNumber(tag, true)` and `eth_getBlockReceipts(tag)` for each, flagged `round_trip`. The mapping is the one input to this module that is not read off the chain, so it is threaded through explicitly rather than inferred: a caller that cannot honour the declaration passes nothing and gets nothing, which is how `consume rlp` stays out of it. What this buys over the recorded corpus is the whole reason to bother. `rpc-compat` replays a `headfcu.json` that points head, safe and finalized at the *same* block, so a client that ignores both fields and answers every tag with the head passes it. Because this chain is generated, the three can name three different blocks, and then only a client tracking three separate pointers can answer all of them. A tag naming a block outside the chain raises `ProjectionError` rather than being emitted. That is the same guard as `_reject_unsatisfiable` and for the same reason: an expectation no client could satisfy reaches a client team as "your response is wrong" and sends them to debug an assertion that was never valid. `_tag_calls` keeps `latest` and `earliest`. The head of the canonical chain is a fact about the chain, not a declaration — it is precisely the tag whose value the harness did not choose — so it stays derived and unflagged.
The declaration goes on the block — `Block(txs=[...], forkchoice_tag="safe")` — rather than on `BlockchainTest`. A field on the test would have to name blocks by index, and an index silently starts naming a different block the moment someone inserts one ahead of it. Blocks are the thing tests grow. The tag also reads where it applies, which is how every other block-scoped property here is written: `exception`, `header_verify`, `expected_post_state`. One tag per block, typed `Literal["safe", "finalized"] | None`, so the two can never name the same block. That restriction is the point rather than a limitation: a chain where two tags coincide is exactly the weakness of the recorded corpus this exists to fix. Four more ways to declare something no client could honour are rejected at model construction, before any transition-tool work: - one tag without the other, which leaves a client inventing behaviour for the missing one; - `finalized` after `safe`, since finalization trails the safe head; - a tag on the head block, which collapses two of the three tags and lets a head-answering client pass; - a tag on a block expected to be rejected, which is not in the chain at all. A fifth, tagging without `pytest.mark.rpc`, is caught at generation by `check_forkchoice_declaration`, called from both fixture paths before the first block is built. Only a marked test emits the `rpc` section, and that section is the only thing that carries the declaration to a consumer, so an unmarked test would tag its blocks and assert nothing. Only `make_hive_fixture` records the triple. `make_fixture` ignores the tags entirely rather than erroring, because a single marked test emits both formats and the RLP one simply has no way to declare them. `_split_blocks_by_phase` clears the tag on every sub-block but the last, alongside the other fields that describe a block's final state.
The final forkchoice update now carries safe and finalized as well as
the head, when the fixture declares them. Only the final one: the blocks
a tag names must already be in the client, and an earlier update would
be overwritten by the next head-only one anyway. "Final" is the last
*valid* payload rather than the last payload, since a test may end on a
block the client is expected to reject.
The declared head is checked against the payload just imported instead
of being trusted. A mismatch means the fixture's tags describe a
different chain, which should fail loudly rather than produce a
confident wrong answer about which block is safe.
Undeclared tests are untouched — both hashes stay zero, which is what a
test that reorgs needs.
`consume rlp` is kept out twice over, because the failure mode is a
false assertion against a conforming client. The filler already omits
the round-trip calls from the RLP fixture, so the primary answer is that
the assertion is not in the artifact. `_replayable` additionally drops
any round-trip call from a fixture that declares no forkchoice state, so
the two facts cannot drift apart. Keying that on the declaration rather
than on the format name states the actual rule: whoever can honour the
declaration may assert what it implies, and nobody else.
That guard is load-bearing, not decorative. Measured with it disabled
and a round-trip call injected into an RLP fixture, go-ethereum v1.17.6
answers `-32000 safe block not found` and a conforming client fails.
A round-trip failure says where its value came from, both in the log
line, which counts the two kinds separately, and in the message:
eth_getBlockByNumber('safe', True) [round trip: value declared by
the harness, not derived]:
number: expected '0x3', got '0x2'
A client team debugging "expected block 2, got block 3" is owed the fact
that block 2 is the answer because this simulator said so.
A three-block chain with `finalized` on block 1, `safe` on block 2 and
the head at block 3. Every block logs its own number, so a client
resolving a tag to the wrong block is caught by the log topic and the
receipt as well as by the block hash.
`tests/paris/` because the tags arrive with the merge, and this is a
property of the Engine API rather than of any EIP.
The distinctness is the entire test. A chain where head, safe and
finalized agree — which is what the recorded `rpc-compat` corpus
replays — is passed by a client that ignores both fields and answers
every tag with the head.
Verified against go-ethereum v1.17.6 under hive. It answers block 1 for
`finalized`, block 2 for `safe` and block 3 for `latest`, so it does
track three separate pointers and the assertion is not vacuous. The
negative control is what establishes that rather than the pass: with the
stored `safe` expectation repointed at the head, the run fails with
1 of 75 RPC expectations failed
eth_getBlockByNumber('safe', True) [round trip: ...]
number: expected '0x3', got '0x2'
which is geth returning block 2 for `safe` while the corrupted fixture
insists on block 3. The same fixture set still passes `consume rlp`,
where the four round-trip calls are absent and the remaining 71
expectations replay unchanged.
Contributor
Author
|
Tried to add eth_simulateV1 but testing against the clients surfaced #3341 as a discrepancy |
Derivation reads its parameters off the chain, so it can only ask questions the chain answers: every log at once, never a filtered subset. A test that wants to pin `eth_getLogs` topic matching must say which filter it means, which makes the call declared rather than enumerated — but the answer is still the specification's, because the chain's logs are already projected and a filter only selects among them. `RPCExpectation` therefore gains a third outcome alongside an error code and an explicit null: `derive_result`, which computes the value at fill time. A result written by hand is still refused, since that would reintroduce the maintained expectation this design exists to avoid, and a computed one does not violate that rule. The set of computable methods is kept beside the dispatch so the validation a test sees at construction and the behaviour at fill time cannot disagree. Filters naming a block hash or a tag are refused rather than guessed at: the hash is unknown until after filling, and a tag resolves against client state rather than the chain. Two encoding faults fixed along the way, both found by a real client rather than by reading. `params` is untyped because a JSON-RPC parameter can be any JSON value, which leaves pydantic no serializer for the `bytes` subclasses a test naturally holds — passing an `Address` failed at fill with a utf-8 decode error. Rendering byte-like parameters as hex fixes that, but must use `bytes(...).hex()`: the project's own byte types override `hex()` to include the prefix, and the first attempt produced `0x0x303103…`, which go-ethereum rejected as an invalid address. Verified against go-ethereum v1.17.6 under hive through both simulators: four filters over a two-emitter chain — by address, by topic, by range, and by topic alternatives — all matching. Widening an address-filtered result from one log to two fails with `expected 2 entries, got 1`, confirming the client genuinely applies the filter.
A caller that already knows the sender -- and whose "transaction" carries no signature at all -- can neither pay for an ECDSA recovery nor satisfy the requirement that the sender be an externally owned account. Both of those are properties of a sender *derived from a signature*, so they are offered as a single concept rather than two independent switches: naming the sender is what removes them, and there is no way to ask for one without the other. Misuse hardening: - The parameter is keyword-only. Fork signatures are not uniform here (Amsterdam's fourth positional is `index`, everywhere else it is `tx_state`), so a positional fifth argument would be an easy way to hand the wrong value to the wrong fork. Requiring the name at every call site removes that class of slip. - It is named for what it is -- an assertion by the caller, not a derivation -- and the docstring states plainly that consensus block execution leaves it `None`. Rejected: refusing a transaction that carries both a signature and an assertion. Every `Transaction` dataclass has r/s/v fields, so "carries no signature" would have to mean a zero-valued signature -- a convention this spec does not otherwise have, and a new error type in 24 forks to enforce it. The signature is simply never read when a sender is asserted, so a contradictory one cannot silently win. Verified: `just static` clean; `just test-tests` 1992 passed / 31 skipped / 4 xfailed / 2 xpassed, identical to the baseline measured on the unmodified tree. Assumed-but-not-yet-verified: that filled fixtures are byte-identical -- that evidence lands in a later commit, once the remaining forks carry the same change. Work in progress: Cancun only. This commit exists to pin the shape.
`process_transaction` computed the top frame's output, its pre-refund gas and its error, then returned `None`. The only way out for the output was the `TransactionEnd` trace event, which exists to serve EIP-3155 traces; nothing promises it keeps carrying that payload. The pre-refund gas figure was never exposed at all. Return a `vm.TransactionResult` instead. `MessageCallOutput` gains `return_data`, matching the field Prague onwards already carries for system contract calls, so the output reaches `process_transaction` without going through the tracer. Pilot fork only; the remaining 23 follow in later commits. Verified: `just static` clean (mypy, ruff, ethereum-spec-lint, vulture, codespell). Behaviour preservation asserted by inspection so far -- nothing reads the new return value and no existing statement moved; fixture diff evidence follows in a later commit.
Adds `eth_getProof` at the schema tier: three subjects per fixture, one per storage shape, plus the address the chain never allocated. No result is stored — the proof is fully determined but we decline to compute it, which is a different reason for this tier than the fee oracles have. Verified against go-ethereum v1.17.6 under hive on both `consume rlp` and `consume engine`, at Prague and Amsterdam, with negative controls at both the parameter and the response.
The bound on how many slots a state read or an account proof may name had never been reached by a filled fixture: no marked test in the corpus had a touched account holding more than one slot, so the branch that drops slots ran only against a post-state written by hand in a unit test. A truncation nothing real reaches is a truncation nobody has checked. Two fixtures straddle the bound, taking their slot counts from the constant so that moving it moves the pair. The writes run downwards, which separates the order the chain wrote the slots in from the order they sort in, and each slot stores its own write position so the surviving set is legible in the fixture rather than inferred. The cap is exported from the serialization package for the test to read.
The proof side of the bound had a unit test and the state-read side had none, which left the branch that drops slots asserted in one of the two places it is written. The pair of counts states where the bound sits rather than merely that it exists, and the surviving set is checked against a storage written in reverse of its sort order, so a derivation reading its keys out of a set keeps the count and fails the order.
No code change. `docs/plans/` is gitignored, so the evidence for the two preceding commits lives here. It also supersedes the design note reading "no marked test has a touched account with more than one storage slot" — two now do, one at the cap and one past it. Client: go-ethereum v1.17.6 (Geth/v1.17.6-unstable-255842b7-20260810, image pulled fresh before the run) under `hive --dev`, on both `consume rlp` and `consume engine`. Determinism. Five independent fills of the pair, four of them under distinct `PYTHONHASHSEED` values, produce byte-identical fixture JSON — one sha256 for the blockchain format and one for the engine format across all five. The order slots are truncated in is therefore a property of the chain rather than of the run, which is what makes a fixture past the cap reproducible at all. Nothing contradicted the reasoning already recorded beside `items()`: the post-state order is the order `storage_writes` recorded the writes in, carried through `apply_diff` into an ordinary dict, with no set anywhere on the path. Truncation, observed. At the cap all 32 slots are asserted and none dropped; one past it the 32 first written are asserted and slot 1 — the last written — is dropped. Because the writes descend and each slot stores its own write position, the surviving set is legible in the fixture rather than inferred. Positive, Prague and Osaka. Both new fixtures replay clean on both simulators at both forks: 17 of 17 blockchain tests at Prague against 15 before, and 127 RPC expectations per new fixture against roughly 62 for an ordinary one. Positive, Amsterdam. 26 of 26 tests report failures on both simulators, 88 failed expectations of 1,923 replayed. A baseline fill with the new file ignored reproduces 82 of 1,663 on both, matching the figure recorded for `eth_getProof`. The delta is exactly six, all of them `eth_getBlockAccessList` answered `-32601` — three per new fixture, for the number, the hash and the `latest` tag. No new failure of any other kind, and none from a state read or a proof. Negative, real client. Corrupting the last slot the cap kept fails on both fixtures and prints go-ethereum's real answer, `0x20`, which is the write position the slot was given. Rewriting the 32-key proof to a wrong `exact` expectation reports `storageProof: expected 1 entries, got 32`, so the truncated request is one the client answers in full and the schema-only tier examines the whole of it. Negative, unit. Slicing one short of the cap fails both new unit tests. Sorting the slots before truncating fails the order test while leaving the count test passing, which is the defect the order test exists for. Relaxing the guard from `>` to `>=` changes nothing observable, since truncating a list to its own length is a no-op — worth stating, because the pair of edges pins where the bound sits and not how it is spelled. Cost. 615 bytes of fixture per asserted slot, flat from 1 slot to 32 and identical at Prague and Amsterdam, so 32 slots cost 19.7 KB. That takes a fixture from the 75 KB an ordinary one occupies to 99 KB at Prague and 138 KB at Amsterdam, leaving the two new cases third and fourth largest in the corpus behind a three-block forkchoice test. The corpus grows 17.1% at Prague and 13.0% at Amsterdam, on two new files each. The split of that 615 is the part worth arguing with. 546 of it is the tagged and untagged `eth_getStorageAt` pair and 69 is the proof key, so 89% of the cost buys the same request repeated against the same account with a different key, and 11% buys a longer `storageProof` array — the one of the two whose response shape actually changes with the count. If the cap exists to bound cost, the two consumers have very different prices and arguably want different bounds; 32 is left as it stands because that is a judgement to make deliberately rather than as a side effect of the first test to reach it.
The counts come from the constant, so the surrounding prose should not spell 32 and 33 back out; it now describes the pair in terms of the cap and records why the suite starts at Prague, as its neighbours do.
Adds an `rpc`-marked test at the cap and one past it, so the branch that drops slots runs against a chain a transition tool produced rather than only against a post-state written by hand. The writes run descending so that write order and sort order differ, which is what makes the ordering property observable: reading the keys out of a set would satisfy the cap while producing a different fixture each run. Five fills under distinct PYTHONHASHSEED values give one sha256, so the order is a property of the chain. Verified against go-ethereum v1.17.6 under hive on both simulators at Prague and Osaka clean, and at Amsterdam with the six added failures accounted for as `eth_getBlockAccessList` -32601.
EIP-7708's specification is a closed list, and eth_simulateV1's traceTransfers is about to synthesize a near-identical log over a scope neither specification states. Write down the EIP-7708 half so the two can be compared against something rather than against whatever a client happened to do. Each case declares a Transfer-topic eth_getLogs filter whose answer is derived from the chain, so a client reporting a movement the EIP excludes has a longer list than the spec computed. Exclusions are paired with a logged transfer: an empty result asserts far less, and the method's OpenRPC oneOf cannot represent one at all.
Adds the priority fee, the base fee burn, a withdrawal, and the same-account clause. The last is the only exclusion carried by the specification proper rather than the rationale — "to a different account" qualifies three of the four bullets — and it is the one a retelling of the EIP loses first. Each fixture was checked to contain the movement it excludes rather than merely to omit a log: a nonzero priority fee credited to the coinbase, a burn well above the default's handful of wei, and a withdrawal that lands an ETH in an account nothing else touches.
Ten tests over the inclusion and exclusion boundary of the transfer log, so that when eth_simulateV1's traceTransfers arrives the two scopes can be compared against something written down. The exclusion the specification actually carries is the "to a different account" clause on three of its four inclusion bullets; the coinbase fee, the base-fee burn and withdrawals are excluded by omission from a closed list and argued only in the Rationale. Nothing normative forbids a client logging those, which is exactly what makes the comparison worth pinning. Verified against go-ethereum, which does implement EIP-7708 and whose scope matches: zero eth_getLogs failures on both simulators.
A precompile is not an account with code, so nothing a caller can say about state moves one: no override shadows it and no write relocates it. Dispatch consulted the module-level `PRE_COMPILED_CONTRACTS` directly, which left rearranging them to editing that dictionary — a process-global edit that outlives the execution that wanted it and reaches every later one. The arrangement now hangs off `BlockEnvironment`, defaulting to the fork's own, so a block on chain never names it and gets exactly what it got before. A caller that wants a precompile somewhere else, or gone, builds its own mapping and hands it to the one environment it cares about; nothing else can see it. Non-escape is structural rather than agreed. The field is typed `Mapping`, and the fork's own arrangement is a read-only view, so there is no longer a global to move a precompile in — the mutation the spike performed now raises. The default is a field default, so no existing construction site had to learn about it. Warming follows the arrangement as dispatch does: a relocated precompile is warm at the address it answers at rather than the one it left. The `PrecompileStart` trace keeps reporting `code_address`, which is where the frame dispatched, so a debugger reads the address that was called. The `is not None` guard on `code_address` is not new behaviour: a creation frame has no code address, and `None` was never a key. It only states in the code what mypy previously inferred from the dictionary. Fixtures are unchanged. Filling 56,327 tests across every fork from tests/frontier/precompiles, identity_precompile, opcodes, istanbul/eip152_blake2, berlin, cancun/eip4844_blobs, prague/eip7702_set_code_tx, prague/eip2537_bls_12_381_precompiles and amsterdam/eip7928_block_level_access_lists gives 3,052 fixture files byte-identical to the same fill before the change.
…rontier Carries the Amsterdam change back through Spurious Dragon: the fork's precompiles become a read-only mapping, `BlockEnvironment` gains a `precompiles` field defaulting to it, and dispatch reads the field rather than the module global. These forks reach the environment one hop further than Amsterdam does, through `evm.message.block_env`, because they still have a `Message`. That is the only difference, and it is the reason Frontier joins Amsterdam in the relocation tests: the two shapes are exercised, not one of them twice. The tests move the identity precompile to an address nothing occupies and check both ends of the move — the new address answers, the old one returns nothing — then run an execution that supplies no mapping and find the fork where it was. A negative control confirms the tests bite: pinning dispatch back to the module global fails the relocation and no-leakage cases at Frontier while the rest keep passing. Fixtures are unchanged: the same 56,327-test fill across every fork gives 3,052 fixture files identical to the pre-change baseline, apart from the commit that `_info.url` names.
…yzantium Carries the same change through Berlin. Byzantium and Istanbul add precompiles, so each fork's mapping stays its own; only the shape around it is shared. Berlin is the first fork to warm the precompile addresses at the start of a transaction, and that warming now follows the arrangement as dispatch does: a relocated precompile is warm where it answers rather than where it left. With no mapping supplied the two are the same set, which is why the access-list fixtures do not move. Fixtures are unchanged: the same 56,327-test fill gives 3,052 fixture files digesting to 77639ead, as they did before the change.
…ondon Carries the same change through Cancun. Cancun adds the point evaluation precompile, so its mapping differs from its predecessor's; the shape around it does not. Fixtures are unchanged: the same 56,327-test fill gives 3,052 fixture files digesting to 77639ead, as they did before the change.
…rague Finishes the change at the last seven forks, so all twenty-four now carry their precompiles on `BlockEnvironment`. Osaka adds a precompile and can disable them all for a delegated call, which is why it joins Frontier and Amsterdam in the relocation tests: three shapes of dispatch, one fork each. The `call` tool now reads the precompiles off the environment the message ran in rather than off the fork module, so its access-list derivation is measured against the arrangement the execution actually saw. A test walks all twenty-four forks and asserts the same three things of each: the environment's `precompiles` field defaults to that fork's own mapping, that mapping refuses to be edited, and neither the interpreter nor `prepare_message` still holds the module global. The precompiles themselves differ from fork to fork; only the way the environment reaches them is shared, so that is what is checked mechanically rather than read. A further test pins the warming: from Berlin on, a relocated precompile is warm at the address it answers at and cold at the one it left, drawn from the same arrangement dispatch reads. Fixtures are unchanged: the same 56,327-test fill gives 3,052 fixture files digesting to 77639ead, as they did before the change.
The relocation cases all addressed the transaction straight at the precompile, which is not how one is normally reached. A fourth case runs the same Amsterdam execution one frame deeper, through a contract that forwards its input, since a child frame carries the parent's block environment and so must see the same arrangement.
The all-forks test already asserts each fork's mapping refuses to be edited, so the parametrized copy of it said the same thing about three of the twenty-four.
Dispatch no longer consults the module-level PRE_COMPILED_CONTRACTS. Each fork's BlockEnvironment carries a precompiles mapping defaulting to the fork's own, so a caller can supply an alternative for one execution without a global to mutate and restore. The argument stands without the RPC that prompted it: BlockEnvironment is already where chain configuration lives, and the precompile set is chain configuration. Dispatch reaching past the environment into a module global was the one place it did that. No fork.py changed in any fork -- a dataclass field default left all 28 construction sites working untouched. Zero consensus change, measured: 3,052 fixtures over 56,327 tests across every fork hash identically before and after.
An access list entry for a contract created during the call cannot save the caller anything. EIP-2929 adds the created address to `accessed_addresses` "immediately (ie. before checks are done to determine whether or not the address is unclaimed)", and the specification does that on the *calling* frame before dispatching the child, so the address is warm on every path — including the one where the creation fails, which the EIP spells out. Declaring it buys warmth already held and charges `ACCESS_LIST_ADDRESS_COST` for it. Measured on the fixture that surfaced this, an Amsterdam factory call: the derived answer was `gasUsed` 0x34719 with the entry and 0x336c5 without, a difference of 4180 — 2900 for the address under EIP-2929 pricing plus 1280 for EIP-7981's access-list bytes. On Cancun, go-ethereum and Nethermind differ by exactly the flat 2400. The list we were returning cost more to use than to ignore. The address is excluded on the same footing as the recipient: the bare address is dropped and its storage slots are not, because a created account's slots are cold and cannot be declared without naming the account. A creation whose init code writes storage therefore keeps its entry, which is also what every client does. Detection needs no fork change. A frame executing init code has no `code_address`, so the tracer that already keeps the settled top-level frame records those frames' targets as it goes. What this does not do is settle the question, and the derivation now says less rather than more. See the accompanying chore commit for the evidence; the short of it is that clients split two and two and no specification decides between them, so a message that creates a contract derives a shape and no value.
Two creations were being treated as contested that no client argues about, and one of them cost an expectation its value. A creation whose init code writes storage is declared for its slots, because a slot cannot be named without naming the account that holds it, and go-ethereum, reth and Nethermind all return that entry — the same address, the same slot, the same gas, to the digit. That case already kept its `exact` tier because the address ends up in the list; the tests now pin it. A message with no recipient at all was not so lucky. Its created address is the message's *recipient*, warmed at the start of the transaction like any other and excluded by name by every client — go-ethereum computes it as `crypto.CreateAddress(args.from(), *args.Nonce)` for that purpose. It was being reported as an omission clients disagree over, which dropped every top-level creation to the `schema` tier for no reason. It is now uncontested. What is left contested is what the evidence actually shows to be: a bare address created by a `CREATE` or `CREATE2` opcode, which go-ethereum and reth omit and Nethermind declares.
`docs/plans/` is gitignored, so the investigation behind the two
preceding commits is recorded here instead. Reproduced against
go-ethereum, reth and Nethermind, all three run rather than read.
----------------------------------------------------------------
# `eth_createAccessList` and a contract created mid-call
**Date:** 2026-08-11
**Status:** reproduced; three clients run empirically; verdict reached
**Verdict:** neither side is non-conformant. The method's semantics are
not specified anywhere, and clients split two and two. But the
specification's answer is the *worse* of the two available answers, and
it is worse by a measurable amount, so it has been changed — and the
expectation has been weakened, because a better answer is still not an
assertable one.
## Summary
Replaying filled fixtures against go-ethereum reports
`accessList: expected 1 entries, got 0` for an auto-enumerated
`eth_createAccessList` whose target is a factory. EELS declares the
address of the contract the call created; go-ethereum does not.
Four things came out of the investigation, and only the first is what
the framing expected.
1. **The gas argument holds, and is measured on both sides.** A created
address is warm the instant it is created, so declaring it saves
nothing and costs `ACCESS_LIST_ADDRESS_COST`. On the Amsterdam
fixture that surfaced this, the derived `gasUsed` is `0x34719` with
the entry and `0x336c5` without — 4180 gas. Against real clients on
Cancun the same gap is exactly 2400. The list EELS returned cost
more to use than to ignore.
2. **go-ethereum has no rule about created addresses.** It excludes the
sender, the callee, the precompiles and EIP-7702 authorities, and
nothing else. A created address is absent only because no watched
opcode named it — put an `EXTCODESIZE` or a `CALL` after the
`CREATE`, or a single `SSTORE` inside the constructor, and
go-ethereum returns it. The note in our own source crediting
go-ethereum with a considered exclusion was wrong about
go-ethereum.
3. **Clients split two and two, verified by running them.** go-ethereum
and reth omit the address; Nethermind declares it; Erigon declares
it by default and can be asked to drop it. The empty-account
playbook — three clients agreeing against the spec — does not apply.
**And the divergence is narrower than the brief said.** It is not
"the spec declares created addresses and clients never do". Give the
constructor a single `SSTORE` and all three clients return the
created address, with its slot, at the same gas — because a slot
cannot be declared without naming the account holding it. Only a
*bare* created address is contested.
4. **Nothing specifies the answer, and upstream knows it.**
execution-apis gives the method one line of prose and no
`description`, and marks three of its own four tests `speconly`.
EIP-2930 mentions "newly created contract" exactly once, in a
rationale paragraph declining to police duplicates. The question was
raised on the geth PR that introduced the method in 2021 and ended
in "not sure".
## 1. Reproduction
```
uv run fill --until Amsterdam -m rpc \
tests/amsterdam/eip7708_eth_transfer_logs/test_rpc_transfer_log_scope.py \
-k create_endowment
```
The derived expectation before this change, from
`rpc_logs_a_create_endowment[fork_Amsterdam-create_opcode_CREATE]`:
```json
{
"method": "eth_createAccessList",
"params": [
{"from": "0xf6c3a9ed…b3ff", "to": "0x4ee05ba5…35f1",
"input": "0x", "value": "0x0",
"gas": "0x1c9c380", "gasPrice": "0x7"},
"0x0"
],
"result": {
"accessList": [
{"address": "0xfdcc0ef1ee030ec294cf99beb5eff203b6aff73d",
"storageKeys": []}
],
"gasUsed": "0x34719"
},
"assertion": "exact"
}
```
`0x4ee0…35f1` is the factory; `0xfdcc…f73d` is the contract it creates,
confirmed by the post-state, which shows it holding the `10**9`
endowment. The call is auto-enumerated by
`_replayed_access_list_calls`, which replays each block's first
transaction; the test declares no RPC check of its own, so the test is
indeed not at fault.
One detail of the original report does not fit and is worth flagging.
`_differences` walks the whole expectation, so a `gasUsed` mismatch
would have been reported alongside the `accessList` one — and
go-ethereum, returning an empty list, should have reported a `gasUsed`
about 4180 lower. Either the quoted message was abbreviated or
something else is going on there. It did not change any conclusion
here, but somebody replaying this should read the full consumer output
rather than trusting the one line.
## 2. Why the entry is there
`generic_create` in `vm/instructions/system.py` does
```python
evm.accessed_addresses.add(contract_address)
```
on the **calling** frame, before the child is dispatched, and never
removes it. EIP-2929 requires precisely that:
> When a `CREATE` or `CREATE2` opcode is called, immediately (ie.
> before checks are done to determine whether or not the address is
> unclaimed) add the address being created to `accessed_addresses`, but
> gas costs of `CREATE` and `CREATE2` are unchanged.
>
> Clarification: If a `CREATE`/`CREATE2` operation fails later on, e.g
> during the execution of `initcode` or has insufficient gas to store
> the code in the state, the `address` of the contract itself remains
> in `access_addresses` (but any additions made within the inner scope
> are reverted).
So the address is warm on every path there is, including the failing
one. `declarable_access_list` reads the settled top-level frame's warm
set and, until now, declared everything in it that no rule warmed.
## 3. The gas consequence, measured twice
**Against the specification.** `declarable_access_list` was temporarily
patched to drop that one address and the fixture refilled, with nothing
else changed.
| | `accessList` | `gasUsed` |
|---|---|---|
| as shipped | 1 entry | `0x34719` = 214809 |
| created address dropped | empty | `0x336c5` = 210629 |
4180 gas, and it decomposes exactly against the Amsterdam constants:
`TX_ACCESS_LIST_ADDRESS` is `COLD_ACCOUNT_ACCESS - WARM_ACCESS` =
`3000 - 100` = 2900, and EIP-7981 adds
`ACCESS_LIST_ADDRESS_FLOOR_TOKENS` = 80 tokens at `TX_DATA_TOKEN_FLOOR`
= 16 = 1280, charged unconditionally in `calculate_intrinsic_cost`
rather than only at the floor. From Berlin through Osaka the same entry
costs the flat 2400 of EIP-2930.
**Against real clients.** On a Cancun genesis shared byte-for-byte by
all three (genesis hash `0x9b9b16d5…67a4`, identical state root), a
call into a factory that creates and does nothing further:
| client | `gasUsed` | created address in list |
|---|---|---|
| `Geth/v1.17.6-unstable-255842b7` | `0xcf25` | absent |
| `reth/v2.4.1-8eb2101` | `0xcf25` | absent |
| `Nethermind/v1.39.3+28cbe2a0` | `0xd885` | **present** |
`0xd885 - 0xcf25` = 2400, exactly `ACCESS_LIST_ADDRESS_COST`. Both
go-ethereum and Nethermind were then asked for `eth_estimateGas` with
the one-entry list supplied explicitly, and both charged about 2400
*more* than with an empty list. The entry does not pay for itself in
any client; there is no cold charge for it to save.
So the answer EELS gave was not merely redundant, it was negative: a
caller who attached the list paid more than a caller who attached
nothing, and the method handed back that inflated figure as the
`gasUsed` to expect. That is the strongest thing that can be said
against it, and it is worth saying plainly.
It is not, however, the same as saying it was *wrong*. "Useless to
include" and "forbidden to include" are different claims, and only the
second would make a client non-conformant. Section 5 keeps them apart.
## 4. What each client actually does
Three clients were run, not read. All queries hit the same genesis, and
a control (`BALANCE` of an unrelated address) confirms every tracer was
live.
| scenario | geth | reth | Nethermind |
|---|---|---|---|
| control: `BALANCE(0x…deadbeef)` | present | present | present |
| `CREATE`, value 1e9, nothing after | **absent** | **absent** | **present** |
| `CREATE`, value 0, nothing after | **absent** | **absent** | **present** |
| `CREATE` then `EXTCODESIZE(created)` | present | present | present |
| `CREATE` then `CALL(created)` | present | present | present |
| `CREATE` whose init code `SSTORE`s | present | present | present |
Only one row diverges, and it is the bare one. The rest are the
important context. In the last three every client returns the created
address, and in the `SSTORE` row they agree on the whole answer — one
entry, address `0xc354…6b5d`, storage key `0x00…07`, `gasUsed`
`0x12e11`. Our own derivation for the same shape produces `gasUsed`
`0x12e11` too, which is about as good a cross-check as this exercise
offers.
So go-ethereum and reth will happily report a created address; their
empty list in the plain case is not because the address is filtered out
but because no opcode they watch named it. Reading the source confirms
it from the other direction:
- **go-ethereum**, `eth/tracers/logger/access_list_tracer.go`. The only
hook is `OnOpcode`, and the opcodes it watches are `SLOAD`, `SSTORE`,
`EXTCODECOPY`, `EXTCODEHASH`, `EXTCODESIZE`, `BALANCE`,
`SELFDESTRUCT` and the four call opcodes. `CREATE` and `CREATE2` are
not among them and there is no `OnEnter`. The exclusion set, built in
`internal/ethapi.AccessList`, is sender, callee, precompiles and
EIP-7702 authorities — the comment reads `// addressesToExclude
contains sender, receiver, precompiles and valid authorizations`. A
created address is in none of it. The one created address
go-ethereum does deliberately exclude is the destination of a
top-level `to == nil` request, via `to =
crypto.CreateAddress(args.from(), *args.Nonce)`.
- **reth**, `revm-inspectors` 0.34.2 `src/access_list.rs`. The same
design: a `step` hook over the same opcode set, `CREATE`/`CREATE2`
absent, and `collect_excluded_addresses` holding sender, callee,
precompiles and 7702 authorities.
- **Nethermind**. Its list comes from the EVM's own EIP-2929 set, not
from an opcode watcher: `TransactionProcessor` reports
`accessedItems.AccessedAddresses`, and
`EvmInstructions.Create.cs` warms the new address into exactly that
set with the comment *"For EIP-2929 support, pre-warm the contract
address in the access tracker"*. Its exclusion list —
`FillAddressesToOptimize` — is sender, recipient, gas beneficiary and
precompiles, and it drops an excluded address only when it carries no
storage keys. A created address is never in it.
- **Erigon** was not run. Its source is the interesting one anyway: it
is the only client that has thought the question through, with an
`optimizeGas` parameter that removes warm addresses — coinbase and
created contracts among them — and the exact economics in a comment:
keep the entry only if `numSlots * (COLD_SLOAD - WARM_READ -
ACCESS_LIST_STORAGE_KEY) > ACCESS_LIST_ADDRESS`, i.e. above 24 slots.
The parameter defaults off on `main`, so Erigon's default answer
declares the created address.
Neither of the two families is optimal in general, and this is the
finding that should temper any triumphalism about the gas argument.
go-ethereum and reth declare an already-warm created address the moment
an `EXTCODESIZE` names it, which costs 2400 for nothing by exactly the
same arithmetic. The `SSTORE` row all three agree on is *also* a loss:
measured with `callTracer`, the transaction goes from 75129 gas to
77329 with that one entry attached — the inner frame saves the 2100
cold-`SSTORE` surcharge while the transaction pays 2400 + 1900 = 4300
for the privilege, a net 2200. Every single access-list entry measured
in this exercise loses money, because everything reachable from a
nested `CREATE` is already warm.
Nobody except Erigon applies the 24-slot rule to the recipient either.
This is a corner of the API where every implementation is approximately
right and none is exactly right, which is the natural consequence of
nobody having specified it.
There is independent confirmation that this is a known, measured mess.
Heimbach et al., *Dissecting the EIP-2930 Optional Access Lists* (FC
2024) find that
> around 20% of these TALs seen in transactions on the mainnet are
> suboptimal, leading to most of these transactions paying more gas
> fees than they would without providing a TAL […] Common mistakes seen
> in the clients include the failure to adequately remove the tx
> sender, tx recipient, and block producer addresses which are
> considered warm automatically. Further, **contracts created in a
> transaction are warm but often still included in the TAL**.
## 5. What is specified: nothing
This is the part that decides the verdict, and it is a negative finding
throughout.
- **execution-apis**, `src/eth/execute.yaml`, is the whole normative
text: `summary: Generates an access list for a transaction.` There is
no `description`. The generated docs page repeats that line. The
vendored `openrpc.json` in this repo agrees: shape only, zero
semantics. Three of upstream's own four tests for the method are
`SpecOnly`, which upstream's generator documents as "the client
response doesn't have to match exactly and is checked for spec
validity only". Hive's `rpc-compat` cannot catch a suboptimal list
either; it asserts structure.
- **EIP-2930** defines what an access list *is* and what it costs —
`ACCESS_LIST_ADDRESS_COST` 2400, `ACCESS_LIST_STORAGE_KEY_COST` 1900
— and says nothing about how a tool should build a recommended one.
It names our case exactly once, in the "Allowing duplicates"
rationale, to decline to legislate: duplicates are permitted because
policing them raises "questions of what to prevent duplication
against: just between two addresses/keys in the access list, between
the access list and the tx sender/recipient/**newly created
contract**, other restrictions?"
- **EIP-2929** settles the *gas* question and only that: the created
address is warmed for free, as quoted in section 2.
- **The question was asked and dropped.** holiman, on
ethereum/go-ethereum#22550, the PR that introduced the method:
> Oh, one more thing: If the `to` is nil, it's a create-tx. The newly
> created contract will wind up in the accesslist, but does _not_
> have to be added by the caller from the outside. Maybe it's not
> something we need to care about. Not sure.
Never followed up.
So: **neither side is wrong.** EELS was not violating anything, and
neither is Nethermind. This is outcome 5 of the brief.
## 6. What was changed, and why anyway
Two changes, both confined to the tooling. Nothing under
`src/ethereum/forks/` was touched, no fork was copied, and nothing here
is consensus-critical: `eth_createAccessList` executes a message that
never entered a block.
**`src/ethereum_spec_tools/evm_tools/call/__init__.py`.** A created
address is now excluded on the same footing as the recipient: the bare
address is dropped, its storage slots are not. A created account's
slots are cold, and a slot cannot be declared without naming the
account that holds it, so a creation whose init code writes storage
keeps its entry — which is also what all four clients do. Detection
needs no fork change: a frame executing init code has no
`code_address`, and the tracer that already keeps the settled top-level
frame now records those frames' targets as it passes them.
`CallResult` gained `undeclared_created` to report the judgement it
made.
This is the smaller change and the easier one to justify. It does not
make us right where we were wrong; it makes the tool give better advice
than it did, on the one criterion — cost — that anybody can measure.
**`packages/testing/src/execution_testing/rpc/serialization/execution.py`.**
`AccessListOutcome` gained `left_a_created_address_out`, and an outcome
that left one out is stored at the `schema` tier. Both asserted fields
turn on the omission — the entry, and the `gasUsed`, which is the gas
*with* the list — so there is nothing left to pin exactly. This is the
change that actually resolves the divergence, and it resolves it by
saying less: go-ethereum, reth and Nethermind all now pass, because
none of them is asserted against.
**The weakening is per shape, not per method**, which matters given
that only one of the six measured shapes diverges. Three creations are
uncontested and keep their exact expectations:
- a created contract whose init code writes storage, declared for its
slots by all three clients;
- the address a top-level `to == nil` message deploys to, excluded by
all three by name;
- a created address some later opcode touches — contested in the sense
that we omit it and everyone else declares it, but that omission
weakens the expectation, which is the conservative direction.
Only a bare created address that nothing else names drops to `schema`.
Across the entire `-m rpc` suite through Amsterdam, that is one
expectation in thirty-two, and it is the new test written for it.
Weakened rather than refused, unlike the delegation case beside it. A
delegated message derives nothing because a list missing a genuinely
useful entry is a wrong answer, not a partial one. Here the answer is
not wrong on anybody's reading, it is merely not agreed, and a shape
check is still worth having on a response nobody else is checking.
Refusing outright would also cost far more: a message that creates a
contract is an ordinary thing for a test to contain.
Three tests were added to
`tests/prague/rpc/test_eth_create_access_list.py`, one per shape: a
creation that stores nothing, whose expectation is a shape; a creation
that writes one slot, whose expectation is a value and matches all
three clients to the gas; and a top-level creation, whose expectation
is an empty list.
## 7. What is left open
- **The Erigon quarter of the client table is read, not run.** Its
default behaviour is inferred from `optimizeGas` defaulting off. Low
stakes — it does not change the split — but it is not first-hand.
- **The `gasUsed` line in the original divergence report.** See section
1. Worth resolving by re-reading the consumer output.
- **The same criticism applies to entries we still emit**, and this is
the honest limit of the gas argument. An already-warm address
carrying fewer than 25 slots is a net loss under Erigon's arithmetic,
and we declare the recipient's slots, and a created contract's, and
so does everybody else — measured at 2200 gas of loss in the
`SSTORE` case. Fixing that would put us alone against every client,
so it is recorded and not acted on. Which means the change made here
is a local improvement in a scheme that is wrong all over, not a
correction of a defect.
- **The causal story for go-ethereum and reth is read, not measured.**
That their empty list follows from `CREATE` being unwatched rather
than from an exclusion rule comes from the tracer source
(`eth/tracers/logger/access_list_tracer.go`, `revm-inspectors`
0.34.2 `src/access_list.rs`), corroborated by the behavioural fact
that both report the address once another opcode names it. It was
not established by instrumenting a client.
- **An upstream report is worth filing, but not as a bug.** The useful
ask is on execution-apis: specify what the method is *for*. If the
answer is "a list that minimises the transaction's gas", then created
addresses, the coinbase and low-slot-count warm addresses all follow
from one sentence, Erigon is already correct, and everybody else has
a clear target. If the answer is "everything the message touched",
Nethermind is correct and the method's `gasUsed` field is
misleading. Either sentence would be worth more than any client-side
patch. There is no execution-apis issue on this today; the only
`createAccessList` items there are a zero-comment stub and two about
fee affordability.
eth_createAccessList has no specified semantics -- one line of summary upstream, no description, and three of upstream's own four tests are speconly. EIP-2930 names this exact case in its rationale and declines to legislate it. Clients split two-two rather than agreeing against us, so this is not the empty-account shape. Only the bare created address diverges: give the constructor one SSTORE and geth, reth and Nethermind agree down to the gas, and so do we. Declaring the address is worse than useless -- EIP-2929 warms it for free, so the entry costs 4180 gas and saves nothing. We drop it, and weaken the expectation per shape rather than per method: five of ninety-seven expectations fall to schema, all genuinely contested.
`set_storage` reaches one slot at a time, so a caller wanting to replace an account's storage wholesale — which is what `eth_simulateV1`'s `state` override asks for — had to reach into `State._storage_tries` itself. Give that operation a name. The account survives; only the storage trie hanging off it is dropped. Unlike `set_storage` this does not insist the account exists, because clearing storage can never leave behind the orphaned storage that assertion guards against.
Port the spike at 043822d79 onto the mechanisms that have landed since it was written, deleting the three process-global substitutions that made it unshippable and the two workarounds that are no longer needed. The sender and the EOA check both fall to `asserted_sender`, which `check_transaction` now takes; precompile relocation is a `Mapping` on the block environment, so the rearrangement dies with the block rather than leaking into the next request; and `process_transaction` returns the return data and the pre-refund gas directly, so neither the `TransactionEnd` scrape nor the intrinsic-plus-tracer reconstruction of `maxUsedGas` survives. That reconstruction was Cancun-shaped and would have been wrong under Amsterdam's two gas pools; each fork now reports its own figure out of its own settlement. Three things the spike got wrong, found while reading it. It duplicated `ForkLoad` rather than using it, so the adapter is now a third the size and inherits the per-fork predicates `t8n` already maintains. It started the ancestor list empty, so `BLOCKHASH` could not see the block the simulation began from and the history contract had no parent hash to be seeded with. And it copied the account at a relocated precompile's address to the target, which go-ethereum does not. Beyond the port: withdrawals in `blockOverrides`, the legacy and 2930 call types, the closing header commitments a post-Cancun block carries, and `traceTransfers` as a no-op from Amsterdam, where EIP-7708 already puts the transfers in the receipts and synthesizing more would report every one of them twice. One thing the port could not have: an unsigned pre-155 transaction. `check_transaction` reads the chain id out of `v` before it consults the asserted sender, so a zero `v` is rejected as a bad signature and the synthesized legacy envelope carries the pre-155 marker instead. It is the same problem `asserted_sender` was introduced to solve, one layer down, and it is recorded at the point of use rather than fixed here.
The 23-of-27 result the assessment reports was measured by a harness in a session scratchpad, and the scratchpad is gone, so the claim has been unverifiable ever since. Rebuild it and commit it. One genesis, rendered twice: the JSON `geth init` reads and the `State` the specification executes against, so both sides demonstrably start from the same place. The harness derives the genesis header itself and checks it against the client's before it trusts anything downstream. Two tiers. The offline one validates every derived answer against the pinned OpenRPC result schema and runs always; the client one starts go-ethereum out of the hive image and compares field by field, and is skipped unless `--simulate-client` is passed. `error.message` is the only exclusion, on execution-apis' own statement that the messages are suggestions. Measured against go-ethereum 1.17.6 (255842b7): 23 of 28 cases match, including the state root and the block hash on every one of them. Of the five that differ, four are the four the assessment already explains. The fifth is new and is ours: with validation off, go-ethereum does not check the nonce in either direction and increments the account's own, so a declared nonce reaches the transaction's RLP and nothing else. `check_transaction` compares unconditionally, so the specification rejects the call. That is the same shape as the sender problem `asserted_sender` was introduced to solve, and the case is kept, marked contested, with the finding written down at it. Twenty-seven of the cases are the shapes the original measurement used, and 23 of those 27 match, which reproduces the number exactly.
… tag The parameter is titled "Block number, tag, or block hash" and only the first two forms were answered. Add the third, in both its bare and its EIP-1898 object shapes, and resolve `safe` and `finalized` where the chain declared them. A hash names the same block a number does, because a filled chain is a single canonical line. That is what makes `requireCanonical` a question about the error rather than the answer: it asks for a failure on a hash off the canonical chain, and the only hashes off this one are hashes the chain never produced, so the refusal says that rather than reporting a plain missing block. `safe` and `finalized` were refused wholesale on the grounds that no chain determines them. True, but the repository already models the declaration: a block carries `forkchoice_tag`, the consumer hands it to the client over `engine_forkchoiceUpdated`, and a call resolved through one is a round trip rather than a derivation. Carry the tag on the call site, resolve against it, and flag the resulting expectation `round_trip` so a consumer that never opens the engine port is not asked to assert it. `pending` stays refused, and now says why in its own terms: a filled chain has a head and no next block, and what a client would build next is a property of its mempool.
Ports the spike with its three process-global mutations deleted, each now having a real mechanism: asserted_sender for the sender and the EOA check, BlockEnvironment.precompiles for relocation, and TransactionResult for return data and pre-refund gas. The conformance harness lives in the repository this time. The previous one produced the 23-of-27 figure and was then lost with a scratchpad, which is why that figure went unreproducible for two days. Re-measured against a newer client: 23 of 28, the same four differences. Also adds destroy_storage to ethereum.state_mpt, without which the state full-replacement override had no public API, and resolves a declared call's block by hash, EIP-1898 object or forkchoice tag.
This was referenced Aug 12, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
(Not reviewed, currently directing claude to prototype)
Related #3339
The large diff is because I vendored in the schema file which is about 21K LOC (see packages/testing/src/execution_testing/rpc/schemas/openrpc.json) -- this file was taken from
execution-apisRelated Issues or PRs
N/A.
Checklist
just static<type>(<area>): <title>, where<type>and<area>come from an appropriateC-<type>, respectivelyA-<area>, label. The title should match the target squash commit message.Cute Animal Picture