Skip to content

[UUID 5/8] UUID aggregation, group-by and distinct#18873

Open
xiangfu0 wants to merge 4 commits into
apache:masterfrom
xiangfu0:uuid-split/05-agg-groupby-distinct
Open

[UUID 5/8] UUID aggregation, group-by and distinct#18873
xiangfu0 wants to merge 4 commits into
apache:masterfrom
xiangfu0:uuid-split/05-agg-groupby-distinct

Conversation

@xiangfu0

@xiangfu0 xiangfu0 commented Jun 29, 2026

Copy link
Copy Markdown
Contributor

What

UUID-aware aggregation and grouping.

Changes

  • DISTINCTCOUNT, DISTINCTCOUNTHLL/HLLPLUS/THETASKETCH/CPCSKETCH/ULL/BITMAP, IntegerTupleSketch, ANY_VALUE
  • no-dictionary single/multi-column group-key generators, UuidToIdMap, BytesDistinctTable, NonScanBasedAggregationOperator

Enables

GROUP BY uuidCol, DISTINCT uuidCol, DISTINCTCOUNT*(uuidCol).

Depends on

#18869 and #18871 (ColumnDataType.UUID).

About this PR / how to review

This is part 5 of 8 splitting #18140 (first-class logical UUID type) into layered PRs, as requested there.

The split is enabled by the v1 design: DataType.UUID has stored type BYTES, so most paths handle it automatically; each PR adds explicit UUID semantics to one subsystem. Head branch lives on xiangfu0/pinot.

This PR is stacked on #18872 (branch uuid-split/04-sse-predicates-cast). Because GitHub PRs to apache must base on master, the Files-changed tab is cumulative (it includes layers 1–5) until the PRs below it merge. Review the commit titled [UUID 5/8] UUID aggregation, group-by and distinct — that is this layer's change. Each parent merge shrinks this diff after a rebase.

Full stack (merge bottom → top)

  1. [UUID 1/8] Add logical UUID type foundation (pinot-spi) #18869 — [UUID 1/8] logical UUID type foundation (pinot-spi)
  2. [UUID 2/8] UUID ingest and segment storage #18870 — [UUID 2/8] UUID ingest and segment storage
  3. [UUID 3/8] UUID result rendering (DataSchema, Arrow/JSON encoders) #18871 — [UUID 3/8] UUID result rendering (DataSchema, Arrow/JSON encoders)
  4. [UUID 4/8] UUID server-side predicates, CAST and transforms #18872 — [UUID 4/8] UUID server-side predicates, CAST and transforms
  5. [UUID 5/8] UUID aggregation, group-by and distinct #18873 — [UUID 5/8] UUID aggregation, group-by and distinct
  6. [UUID 6/8] UUID multi-stage engine (planner + runtime) #18874 — [UUID 6/8] UUID multi-stage engine (planner + runtime)
  7. [UUID 7/8] UUID scalar UDFs and partitioning #18875 — [UUID 7/8] UUID scalar UDFs and partitioning
  8. [UUID 8/8] UUID integration tests, benchmarks and docs #18876 — [UUID 8/8] UUID integration tests, benchmarks and docs

After #18869 lands, PRs 2/4/7 (#18870, #18872, #18875) only depend on it and can be reviewed in parallel; PRs 5/6 (#18873, #18874) also need #18871; PR 8 (#18876) needs #18874.

Full feature description, v1 design contract, scope exclusions, and benchmark numbers: #18140.

@codecov-commenter

codecov-commenter commented Jun 30, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 52.32029% with 524 lines in your changes missing coverage. Please review.
✅ Project coverage is 64.86%. Comparing base (b6c3ca6) to head (c5f4959).
⚠️ Report is 13 commits behind head on master.

Files with missing lines Patch % Lines
...ot/common/request/context/RequestContextUtils.java 55.30% 48 Missing and 11 partials ⚠️
...java/org/apache/pinot/common/utils/DataSchema.java 55.55% 36 Missing and 8 partials ⚠️
...ator/transform/function/CastTransformFunction.java 34.92% 40 Missing and 1 partial ⚠️
...sform/function/ScalarTransformFunctionWrapper.java 17.07% 34 Missing ⚠️
...pby/NoDictionarySingleColumnGroupKeyGenerator.java 42.00% 23 Missing and 6 partials ⚠️
.../function/DistinctCountULLAggregationFunction.java 0.00% 26 Missing ⚠️
...perator/query/NonScanBasedAggregationOperator.java 20.00% 22 Missing and 2 partials ⚠️
...ion/DistinctCountCPCSketchAggregationFunction.java 0.00% 24 Missing ⚠️
...upby/NoDictionaryMultiColumnGroupKeyGenerator.java 26.66% 19 Missing and 3 partials ⚠️
...nction/DistinctCountBitmapAggregationFunction.java 24.00% 16 Missing and 3 partials ⚠️
... and 30 more
Additional details and impacted files
@@             Coverage Diff              @@
##             master   #18873      +/-   ##
============================================
- Coverage     64.89%   64.86%   -0.03%     
  Complexity     1347     1347              
============================================
  Files          3398     3399       +1     
  Lines        212550   213491     +941     
  Branches      33503    33709     +206     
============================================
+ Hits         137927   138488     +561     
- Misses        63481    63802     +321     
- Partials      11142    11201      +59     
Flag Coverage Δ
custom-integration1 100.00% <ø> (ø)
integration 100.00% <ø> (ø)
integration1 100.00% <ø> (ø)
integration2 0.00% <ø> (ø)
java-21 64.86% <52.32%> (-0.03%) ⬇️
temurin 64.86% <52.32%> (-0.03%) ⬇️
unittests 64.86% <52.32%> (-0.03%) ⬇️
unittests1 56.82% <40.40%> (-0.06%) ⬇️
unittests2 37.20% <19.74%> (-0.06%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@xiangfu0 xiangfu0 force-pushed the uuid-split/05-agg-groupby-distinct branch 7 times, most recently from b5e8bc6 to 64cb941 Compare July 7, 2026 07:08
@xiangfu0 xiangfu0 force-pushed the uuid-split/05-agg-groupby-distinct branch 5 times, most recently from 4a78138 to dac264f Compare July 7, 2026 21:48
xiangfu0 added 4 commits July 8, 2026 01:05
Part 2/8 of splitting apache#18140 (logical UUID type). Rebased onto master (now includes the merged apache#18869 type foundation); stacked on 92a0d98.

Downstream references use the UuidKey class merged in apache#18869.
Part 3/8 of splitting apache#18140 (logical UUID type). Rebased onto latest master; stacked on uuid-split/02-ingest-storage.

Downstream references use the UuidKey class merged in apache#18869.
Part 4/8 of splitting apache#18140 (logical UUID type). Rebased onto latest master; stacked on uuid-split/03-result-rendering.

Downstream references use the UuidKey class merged in apache#18869.
Part 5/8 of splitting apache#18140 (logical UUID type). Rebased onto latest master; stacked on uuid-split/04-sse-predicates-cast.

Downstream references use the UuidKey class merged in apache#18869.
@xiangfu0 xiangfu0 force-pushed the uuid-split/05-agg-groupby-distinct branch from dac264f to c5f4959 Compare July 8, 2026 08:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants