Skip to content

[FLINK-40191][python] Add aggregation support to PyFlink DataFrame API - #28937

Open
auroflow wants to merge 2 commits into
apache:masterfrom
auroflow:codex/FLINK-40191-dataframe-aggregation
Open

[FLINK-40191][python] Add aggregation support to PyFlink DataFrame API#28937
auroflow wants to merge 2 commits into
apache:masterfrom
auroflow:codex/FLINK-40191-dataframe-aggregation

Conversation

@auroflow

@auroflow auroflow commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

What is the purpose of the change

This pull request adds global and grouped aggregation support to the PyFlink DataFrame API, as described in FLINK-40191.

It allows users to aggregate an entire DataFrame with DataFrame.agg(), or group rows with DataFrame.group_by() and aggregate them with GroupedDataFrame.agg().

Brief change log

  • Add DataFrame.agg() for global aggregations.
  • Add DataFrame.group_by() and GroupedDataFrame.agg() for grouped aggregations.
  • Export GroupedDataFrame from pyflink.dataframe.
  • Add API documentation and usage examples.

Verifying this change

This change added tests and can be verified as follows:

  • Added unit tests for global and grouped aggregation schemas and output ordering.
  • Added unit tests for Python argument validation and planner-delegated validation.
  • Added a batch integration test covering grouped sum and count aggregations.

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): no
  • The public API, i.e., is any changed class annotated with @Public(Evolving): yes
  • The serializers: no
  • The runtime per-record code paths (performance sensitive): no
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
  • The S3 file system connector: no

Documentation

  • Does this pull request introduce a new feature? yes
  • If yes, how is the feature documented? PyDocs and Python API docstrings

Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

Generated-by: OpenAI Codex (GPT-5.6)

Add global and grouped aggregation APIs to PyFlink DataFrame, including planner-backed validation tests, one batch execution test, and reference documentation.

Generated-by: OpenAI Codex (GPT-5.6)
@flinkbot

flinkbot commented Aug 7, 2026

Copy link
Copy Markdown
Collaborator

CI report:

Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

Show positional and named aggregations together and document the returned DataFrame schemas.

Generated-by: OpenAI Codex (GPT-5.6)
@auroflow
auroflow marked this pull request as ready for review August 7, 2026 03:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants