Skip to content

[Rust] Add persistent stream proto - #761

Open
elenagaljak-db wants to merge 1 commit into
mainfrom
stack/eos-proto
Open

[Rust] Add persistent stream proto#761
elenagaljak-db wants to merge 1 commit into
mainfrom
stack/eos-proto

Conversation

@elenagaljak-db

@elenagaljak-db elenagaljak-db commented Aug 20, 2026

Copy link
Copy Markdown
Collaborator

What changes are proposed in this pull request?

This PR adds the in-development protobuf contract used by persistent Zerobus ingestion streams. It contains only the wire-level API.

The contract adds:

  • PersistentStream, a bidirectional RPC whose first message creates a durable stream or resumes one by stream_id.
  • ResumeIngestStreamRequest, including the protobuf descriptor and required record_type used to validate a resumed stream against its original configuration.
  • ResumeIngestStreamResponse.last_committed_offset, the server watermark from which the client continues at last_committed_offset + 1.
  • Persistent envelopes for create, resume, ingestion, acknowledgements, and server close signals.
  • The server-side RetireStream contract; a public Rust retirement API is intentionally deferred.

The shapes and field numbers match the current Shinkansen server contract. The PR also records this API-contract addition in rust/NEXT_CHANGELOG.md.

Stack

  1. [Rust] Add persistent stream proto #761 — protobuf contract
  2. [Rust] Add persistent gRPC transport #762 — gRPC transport and recovery engine
  3. [Rust] Expose persistent stream API #763 — public Rust API
  4. [Rust] Test persistent stream resume #764 — stateful mock and integration tests
  5. [Rust] Document persistent streams #765 — documentation and runnable example

How is this tested?

The protobuf is compiled by the Rust SDK build. The complete create, ingest, flush, and resume flow is exercised in #764.

Signed-off-by: elenagaljak-db <elena.galjak@databricks.com>
@elenagaljak-db
elenagaljak-db requested a review from a team August 20, 2026 14:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant