Skip to content

feat(firestore): Support for Firestore pipelines API #8931

Open
russellwheatley wants to merge 135 commits intomainfrom
firestore-pipelines
Open

feat(firestore): Support for Firestore pipelines API #8931
russellwheatley wants to merge 135 commits intomainfrom
firestore-pipelines

Conversation

@russellwheatley
Copy link
Member

@russellwheatley russellwheatley commented Mar 13, 2026

Description

Support for Firestore pipelines API

  • Adds a new public Firestore Pipelines API at @react-native-firebase/firestore/pipelines so consumers can build pipelines with db.pipeline() and run them with execute(...).
  • Keeps pipeline construction in JS and defers all native work until a single pipelineExecute(...) bridge call at execution time.
  • Treats pipelines as a cross-platform feature for Android, iOS, and web, with platform-specific parsing/building/execution behind the same serialized payload.

Public JS & runtime shape

  • import '@react-native-firebase/firestore/pipelines' installs pipeline() onto the Firestore runtime prototype as a side effect. Similar to how it works on firebase-js-sdk.
  • The new entrypoint exports the pipeline builder types, execute helper, result types, stage option types, and a large set of expression/helper functions aligned with the JS SDK naming.
  • Source builders now support collection, collectionGroup, database, documents, and createFrom(query).
  • Pipeline instances are immutable; each stage append returns a new pipeline.
  • JS runtime normalizes overloads and option aliases before serialization, so native sees a consistent shape even when the public API has overloads.
  • Runtime guards prevent invalid combinations early, especially:
    • mixing pipelines/queries/references from different Firestore instances
    • invalid documents(...) inputs
    • union(...) self-cycles or cross-instance pipelines

pipelineExecute() JS -> native contract

  • packages/firestore/lib/types/internal.ts is the source of truth for the data sent over the wire to native.
  • JS no longer sends loose dictionaries; it serializes explicit node types such as:
    • field expressions
    • constant expressions
    • function expressions
    • orderings
    • aliased expressions
    • aliased aggregates
  • The request shape is:
    • pipeline.source: one of collection | collectionGroup | database | documents | query
    • pipeline.stages: ordered { stage, options }[]
    • options: currently indexMode?: 'recommended' and rawOptions?: Record<string, unknown>
  • Query-backed sources are serialized as a typed query payload (path, queryType, filters, orders, options) rather than as an already-built native query.
  • The response shape is executionTime plus results[], which are rehydrated back into PipelineSnapshot / PipelineResult.
  • Reviewer implication: the serialization boundary in JS is now a major contract. Future platform work should preserve payload compatibility rather than inventing platform-local shapes.

Native architecture by platform

  • Android now follows a clear parse -> build native SDK objects -> execute -> serialize snapshot flow.
  • Android initially relied on brittle raw map plumbing for most pipeline work; now the parser produces typed DTOs and the node builder lowers those to Firestore SDK pipeline objects.
  • iOS now follows the same broad separation, but via parse -> bridge factory/node builder -> PipelineBridge.execute -> snapshot serializer.
  • Web was refactored to mirror native conceptually: parse serialized payload, rebuild a JS SDK pipeline, execute with the web SDK, then serialize the snapshot back into RNFB's internal response shape.

Parsing & query building

  • Native parsers validates the serialized payload up front instead of coercing deep raw dictionaries during execution which it was doing initially which made it nigh on impossible to reason with.
  • Query source support was added so createFrom(query) can be serialized in JS and rebuilt natively/web later.
  • Android rebuilds query sources using existing query infrastructure, then feeds that query into pipelineSource.createFrom(...).
  • iOS also rebuilds query sources through existing query infrastructure, but does so through typed parser DTOs plus bridge-factory glue.

Known limitations & current guards

  • execute({ indexMode, rawOptions }) is rejected in JS on all platforms, because the execute-option surface is not available yet (It just throws an error on all platforms).
  • iOS also rejects pipeline.source.rawOptions for source builders because the linked iOS pipeline bridge does not currently expose source options.
  • iOS keeps a JS-side unsupported-function allowlist/denylist guard for pipeline functions that the current iOS lowering/runtime path does not yet support.
  • That guard is intentional and should not be removed until the linked iOS native runtime is verified and the iOS node builder actually supports those functions. The public API shape is ahead of full iOS/native support in a few places; some options/functions exist in types and serialization but are intentionally blocked on native platforms.

Snapshot & result handling

  • Native and web responses are normalized into the same internal snapshot/result format before becoming public PipelineSnapshot / PipelineResult objects.
  • Extra work was added around timestamp/result serialization so returned values preserve Firestore-like types consistently.
  • iOS snapshot serialization includes fallback metadata handling for cases where the bridge does not surface all document metadata directly.

Tests

  • The branch adds unit tests for runtime serialization/guards, web rebuild/execute behavior, and native parity checks.
  • Large E2E coverage was added for source builders, createFrom(query), documents source, stage execution, expression behavior, unhappy paths, and platform-specific option failures.

Why the compare-types script changed

  • The compare-types tooling was extended so pipeline types can be checked as a separate public surface, not mixed into the main Firestore modular API.
  • A dedicated firestore-pipelines SDK snapshot/config was added so CI can detect drift between RNFB pipeline types and the Firestore JS SDK pipeline types.
  • The parser/comparer was hardened to avoid noisy false positives from formatting-only differences, inline comments, and class-vs-interface declaration differences (This is the reason why about 20 different Firestore diffs were included, the parser was ignoring firebase-js-sdk class v react-native-firebase interfaces, not it detects them and checks).
  • The registry now supports multiple primary/support declaration files, which was needed because the pipelines API is exported from its own path and re-exports several supporting types.

Related issues

Release Summary

Checklist

  • I read the Contributor Guide and followed the process outlined there for submitting PRs.
    • Yes
  • My change supports the following platforms;
    • Android
    • iOS
    • Other (macOS, web)
  • My change includes tests;
    • e2e tests added or updated in packages/\*\*/e2e
    • jest tests added or updated in packages/\*\*/__tests__
  • I have updated TypeScript types that are affected by my change.
  • This is a breaking change;
    • Yes
    • No

Test Plan


Think react-native-firebase is great? Please consider supporting the project with any of the below:

… and option types

- Add arithmetic/constant helpers: constant, add, subtract, divide, multiply, documentId
- Add aggregate helpers: sum, count, average, arrayAgg, countDistinct, first, last
- Add math/conditional helpers: abs, ceil, floor, mod, round, conditional, sqrt, not, ifAbsent, ifError
- Add string helpers: toLower, toUpper, trim, substring
- Add concat, currentTimestamp
- Add option/type exports: StageOptions, AliasedAggregate, AliasedExpression, AddFieldsStageOptions, AggregateStageOptions, ExpressionType
- Match add() to SDK (two-arg only, no rest params)
… helpers

- Declare ceil/round with (string) before (Expression) to match JS SDK
- Add arrayAggDistinct, arrayConcat, arrayGet, arrayLength, arraySum
- Reduces firestore-pipelines missing count (86→81) and different shape (4→2)
…re:types)

- Add length(fieldName) and length(expression) to pipelines stage API
- Export stage option types from pipelines index (OneOf, *StageOptions, PipelineExecuteOptions)
- Document 47 firestore-pipelines differences in compare-types config (21 extra in RN, 26 different shape)
- Restore DatabaseStageOptions as StageOptions & {} to match SDK
@russellwheatley russellwheatley marked this pull request as ready for review March 20, 2026 16:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant