Skip to content

[Rust] Add persistent gRPC transport - #762

Open
elenagaljak-db wants to merge 1 commit into
stack/eos-protofrom
stack/eos-grpc-core
Open

[Rust] Add persistent gRPC transport#762
elenagaljak-db wants to merge 1 commit into
stack/eos-protofrom
stack/eos-grpc-core

Conversation

@elenagaljak-db

@elenagaljak-db elenagaljak-db commented Aug 20, 2026

Copy link
Copy Markdown
Collaborator

What changes are proposed in this pull request?

This PR implements the internal gRPC transport and recovery machinery for persistent streams. It deliberately does not expose a new public SDK type; that surface is added in #763.

The sender, receiver, connection, and supervisor tasks are shared between ephemeral and persistent streams through a typed transport seam. Persistent streams use the dedicated RPC while ephemeral behavior remains unchanged.

Persistent behavior includes:

  • Create once, then resume the same stream_id after recovery.
  • Stable logical offsets on the persistent wire protocol.
  • New ingestion beginning at last_committed_offset + 1 after process-level resume.
  • Landing-zone reconciliation for the lost-ack race where the server committed records before the client disconnected.
  • Local completion of reconciled waiters and callbacks, with only offsets above the watermark resent.
  • Existing backpressure, flush, close, callback, credential-refresh, and initial-retry behavior shared across transports.

The implementation remains behind eos. Its internal transport/recovery addition is recorded in rust/NEXT_CHANGELOG.md.

Stack

  1. [Rust] Add persistent stream proto #761 — protobuf contract
  2. [Rust] Add persistent gRPC transport #762 — gRPC transport and recovery engine
  3. [Rust] Expose persistent stream API #763 — public Rust API
  4. [Rust] Test persistent stream resume #764 — stateful mock and integration tests
  5. [Rust] Document persistent streams #765 — documentation and runnable example

How is this tested?

  • cargo check -p databricks-zerobus-ingest-sdk --features eos
  • cargo test -p databricks-zerobus-ingest-sdk --features eos --lib — 156 tests passed
  • End-to-end persistent coverage is added in [Rust] Test persistent stream resume #764.

Signed-off-by: elenagaljak-db <elena.galjak@databricks.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant