Feat/prover db 3 consumer#188
Open
rishitesh-snt wants to merge 18 commits intomainfrom
Open
Conversation
Adds the operator-facing wiring for the upcoming prover-db-indexer pallet: a `--prover-db-url` CLI flag that, at startup, seeds the URL into the OCW persistent local-storage key `prover_db_indexer::prover_db_url` before the first block is authored. Also surfaces a clear error if `--prover-db-url` is set without `--enable-offchain-indexing=true`, since the pallet's on_finalize writes would otherwise be silently dropped. The storage key is kept as a local literal in `node/src/service.rs` so this change is independent of the pallet, which is added in a follow-up PR. The literal must stay in sync with `pallet_prover_db_indexer::PROVER_DB_URL_KEY` once that pallet lands; the next PR refactors the node to import it. The flag is inert until the pallet ships, so this PR cannot affect the running system.
1.69.0Bug Fixes
Features
Reverts
|
- Hoist `PROVER_DB_URL_KEY` into `sxt-core` so node and the prover-db-indexer pallet share one canonical key (with `/` as the domain separator, matching substrate convention). - Tighten the CLI doc since it surfaces in `--help`; drop the internals. - Enable clap `env` feature so `PROVER_DB_URL` env var also configures it. - Drop the eprintln! warning when offchain indexing is off without `--prover-db-url` — that's the typical case, not worth shouting. - Pass `&Cli` into `new_full_base` instead of an owned URL, so future options can be threaded through without further signature changes. - Hoist `use codec::Encode` to the file's import block.
rishitesh-snt
force-pushed
the
feat/prover-db-3-consumer
branch
from
18e54e3 to
172ab5e
Compare
Standardize on "prover-db indexer URL" across `sxt_core`, the node CLI, and the service-side seeding helper. Drop the pallet-internal references (`on_finalize`, `sp_io::offchain_index::set`) from the runtime check — that scaffolding lands in follow-up PRs and shouldn't leak into PR1.
rishitesh-snt
force-pushed
the
feat/prover-db-3-consumer
branch
from
172ab5e to
5fdbcb6
Compare
Per review: sxt-core groups items by the pallet they belong to (with the exception of `IdentLength`/`ByteString`). Move `PROVER_DB_URL_KEY` out of the top-level into a new `prover_db_indexer` module to follow that pattern. The node now imports it as `sxt_core::prover_db_indexer::PROVER_DB_URL_KEY`.
Introduces `pallet-prover-db-indexer`, an offchain worker that will
forward table lifecycle and `pallet_indexing` quorum events to an
external prover-db indexer over HTTP.
This PR ships only the **producer** side (`on_finalize`):
- Reads `frame_system::Events`, downcasts via `RuntimeEvent: TryInto<...>`
(no SCALE re-encoding), extracts `pallet_tables::{SchemaUpdated,
TablesCreatedWithCommitments, TableDropped}` and
`pallet_indexing::QuorumReached` into a `BlockIndex` of `BlockEvent`s.
- Persists non-empty `BlockIndex`es to offchain storage keyed by block
number using `sp_io::offchain_index::set` (a no-op unless the node
was started with `--enable-offchain-indexing=true`).
The **consumer** half (HTTP forwarding via `offchain_worker`) lands in
the follow-up PR; until then `offchain_worker` is a no-op stub. Because
producer writes are gated by `--enable-offchain-indexing` at the node
level — and the operator-facing flag added in the previous PR refuses
to seed `--prover-db-url` without it — landing this PR alone cannot
affect a running system.
Wiring:
- Workspace member + workspace dep entry in root `Cargo.toml`.
- Runtime registers the pallet at `pallet_index(111)` with
`Config<native_api::Api>`.
- `node/src/service.rs` switches its URL-key literal over to
`sxt_runtime::pallet_prover_db_indexer::PROVER_DB_URL_KEY`, replacing
the local placeholder introduced in the previous PR.
`#[pallet::hooks]` expands into code that uses `map_err` over `inspect_err`, which `clippy --all-features` flags under `clippy::manual_inspect`. The hit is in macro-generated code, not in our hook bodies, so allow it on the pallet module's existing allow list.
Replaces the no-op `offchain_worker` stub with the OCW HTTP-forwarding
consumer that drains the producer's offchain DB queue and posts
protobuf payloads to the configured prover-db indexer.
The OCW path:
1. Reads the indexer URL from offchain local storage
(`PROVER_DB_URL_KEY`); skips the round if unset.
2. Asks the server for its last checkpoint via `/v1/get_last_checkpoint`
— server is the sole source of truth, so we never diverge from
what's actually been committed.
3. Walks blocks `cursor+1..=tip` (capped at `MAX_BLOCKS_PER_INVOCATION`),
forwards each `BlockEvent` (`create_table` / `drop_table` /
`put_batches`), checkpoints on the server, deletes consumed
entries from the offchain DB.
Adds:
- `src/http_client.rs` — `sp_io::offchain::http` POST helpers and
`/v1` endpoint wrappers (create_table, drop_table, put_batches,
checkpoint, get_last_checkpoint).
- `proto/prover-db.proto` + `build.rs` — wire-format definitions
generated via `prost-build`; included in `lib.rs` as `mod proto`.
- `offchain_index::{read, clear}` — consumer-side helpers to load
and delete entries the producer wrote.
- `src/{mock.rs, tests.rs}` — mock runtime + 4 integration tests
covering the OCW path: skip-when-unconfigured, single-block forward
with deletion, empty-block checkpointing, server-checkpoint resume,
multi-block in-order processing.
- Cargo deps: `prost`, `prost-build`, and the dev-deps the mock
runtime needs (`pallet-permissions`, `pallet-commitments`, etc.,
`parking_lot`, `proof-of-sql-commitment-map`).
Surface OCW HTTP traffic and failures so operators can diagnose issues against a real prb-service without rebuilding the node: - New `endpoint()` helper trims a trailing `/` on the base URL before joining a path, so a `url::Url`-derived base (which normalizes to end in `/`) no longer produces `host:port//v1/...` and a 404. - Capture the response body on non-200 and surface it inline in the warn log + the `Status` error variant, so 4xx/5xx reasons (route not found, bad payload) are visible. - Add warn logs on Request::send / try_wait / response errors, carrying the URL and the underlying offchain http error variant. - Add debug logs at consumer-round start, per POST request, and per POST response (status + body size). - Include URL in the existing get_last_checkpoint warn so the operator can see exactly which endpoint failed.
rishitesh-snt
force-pushed
the
feat/prover-db-3-consumer
branch
from
d6690b1 to
d8585ca
Compare
tlovell-sxt
reviewed
May 4, 2026
Comment on lines
+69
to
+81
| /// `host:port//v1/...` when callers pass a `url::Url` (which normalizes | ||
| /// to include a trailing `/` on the path component). | ||
| fn endpoint(base_url: &str, path: &str) -> String { | ||
| format!("{}{}", base_url.trim_end_matches('/'), path) | ||
| } | ||
|
|
||
| /// Fetch the last checkpoint from the server. | ||
| pub fn get_last_checkpoint(base_url: &str) -> Result<Option<u64>, Error> { | ||
| let url = endpoint(base_url, "/v1/get_last_checkpoint"); | ||
| let body = post(&url, &[])?; | ||
| let resp = proto::GetLastCheckpointResponse::decode(body.as_slice())?; | ||
| Ok(resp.has_checkpoint.then_some(resp.sequence_number)) | ||
| } |
Contributor
There was a problem hiding this comment.
I would have thought using url.join("v1/get_last_checkpoint") would work as you'd expect, based off the docs
| arrow_schema, | ||
| key, | ||
| }; | ||
| let url = endpoint(base_url, "/v1/create_table"); |
Contributor
There was a problem hiding this comment.
It seems like these extra &'static str should just be constants in this file.
| } | ||
|
|
||
| Ok(response_body) | ||
| } |
Contributor
There was a problem hiding this comment.
This PR is still kinda big. Would you be opposed to splitting it into one that adds the http client and one that uses it in the OCW?
Comment on lines
+211
to
+215
| log::warn!( | ||
| target: "prover_db_indexer", | ||
| "POST {} -> status {}; body ({} bytes): {}", | ||
| url, status, response_body.len(), snippet, | ||
| ); |
Contributor
There was a problem hiding this comment.
I know I mentioned this on the last PR already, but in this one too the logging should probably be done with sp_tracing
| /// Fetch the last checkpoint from the server. | ||
| pub fn get_last_checkpoint(base_url: &str) -> Result<Option<u64>, Error> { | ||
| let url = endpoint(base_url, "/v1/get_last_checkpoint"); | ||
| let body = post(&url, &[])?; |
Contributor
There was a problem hiding this comment.
Are these really all POST requests? I'm somewhat surprised this one isn't a GET but maybe it's because there's some HTTP -> GRPC translation going on that assumes post?
| ); | ||
|
|
||
| // 2. Current block number. | ||
| let current_block: u64 = polkadot_sdk::frame_system::Pallet::<T>::block_number() |
Contributor
There was a problem hiding this comment.
Can't we just get this passed in by offchain_worker? It is called with a block number
| record_batch: entry.data.clone(), | ||
| }], | ||
| ) | ||
| .map_err(|_| "put_batches failed")?; |
Contributor
There was a problem hiding this comment.
I would prefer the errors of these two functions to be expressed with a Snafu enum, and to .to_string() them as needed
| start, end, cursor, current_block, | ||
| ); | ||
|
|
||
| for block_num in start..=end { |
Contributor
There was a problem hiding this comment.
So here's a question - Theoretically an offchain workers keep getting spawned per block-ish. If a previous offchain worker is not finished forwarding, and a new one begins, does this logic do anything to prevent them from submitting out of order? Does it need to (i.e., does the PRB reject anything below MAX(META_ROW_NUMBER), or does it just reject duplicates)?
…trait - sxt-core: add EventCapture trait, types, offchain key helpers - pallet-prover-db-indexer: drop on_finalize, impl EventCapture with on-chain manifest - pallet-tables: Config::EventCapture; capture at SchemaUpdated/TableDropped - pallet-indexing: Config::EventCapture; capture at QuorumReached - runtime: wire EventCapture, add MaxEventsPerBlock=2048 - mocks: type EventCapture = () in tables/indexing/rewards/system_tables/smartcontracts - README: drop on_finalize/CLI-specific/server-side sections - node/cli: --prover-db-url help notes offchain-indexing/worker flags
…ucer # Conflicts: # node/src/cli.rs # sxt-core/src/lib.rs # sxt-core/src/prover_db_indexer.rs
…ain high-water-mark - sxt-core: rename key_for_manifest -> key_for_high_water; value semantics shift from Vec<u32> to u32 - pallet-prover-db-indexer: drop CurrentBlockManifest storage, MaxEventsPerBlock Config type, and on_initialize hook; capture_events writes event blob + hwm only - runtime: drop MaxEventsPerBlock parameter_types and Config wiring - pallet Cargo.toml: drop log dep (overflow warn path is gone)
…allet wrapper - pallet: Pallet<T, I> -> Pallet<T>, Config<I> -> Config; instance was vestigial since the supertrait bound on pallet_indexing::Config<I> was dropped - delete native_pallet.rs: no instance to pin, runtime can reference Pallet<Runtime> directly - Cargo.toml: drop native-api dep - runtime: drop ::native_pallet:: path segment in three places
Drop consumer/OCW, HTTP wire contract, OCW URL, and OCW scheduling sections — those land with the consumer PR.
…e capture, hwm probe Merge feat/prover-db-2-producer (extrinsic-time capture via EventCapture trait) into PR 3, then rewrite the consumer to read the new offchain key shape. - lib.rs: drop on_finalize/extract_block_index/try_extract_*; keep offchain_worker hook calling run_consumer; rewrite forward_block to read key_for_high_water then probe key_for_event(block, 0..=hwm); split forwarding loop into forward_block (per-block) + forward_events (per-extrinsic) - offchain_index.rs deleted (BlockIndex/key_for_block scheme retired) - offchain_consumer.rs added: OCW-only read_high_water/read_events/clear_high_water/clear_events helpers - Cargo.toml: drop pallet-tables/pallet-indexing/native-api main deps; keep prost/log/build-deps and dev-deps - mock.rs: minimal frame_system + pallet only (Config no longer pulls in tables/indexing/balances/staking/etc.) - tests.rs: seed offchain DB via key_for_event + key_for_high_water; add a test exercising the multiple-extrinsics-in-one-block probe path
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rationale for this change
Replaces the offchain_worker stub with the OCW HTTP-forwarding consumer that drains the producer queue, asks the server for its last checkpoint, and posts protobuf payloads (create_table / drop_table /
put_batches / checkpoint). Adds http_client.rs, the prover-db.proto wire format generated via prost-build, and 4 integration tests covering the OCW path.