Skip to content

[opt](nereids) Reuse identical uncorrelated scalar subqueries referencing the same CTE - #66781

Open
LLLLLL302311 wants to merge 1 commit into
apache:masterfrom
LLLLLL302311:opt-reuse-cte-scalar-subquery
Open

[opt](nereids) Reuse identical uncorrelated scalar subqueries referencing the same CTE#66781
LLLLLL302311 wants to merge 1 commit into
apache:masterfrom
LLLLLL302311:opt-reuse-cte-scalar-subquery

Conversation

@LLLLLL302311

Copy link
Copy Markdown

What problem does this PR solve?

Problem Summary:

When a query contains many structurally-identical uncorrelated scalar subqueries
that reference the same CTE (e.g. (SELECT dt FROM deal_dt) repeated hundreds of
times), SubqueryToApply generated one LogicalApply per occurrence. This blew up
the Nereids optimization search space and caused the planner to time out
(Nereids cost too much time), while these queries planned fine on the legacy optimizer.

This PR groups reusable scalar subqueries within the same LogicalFilter/LogicalProject
by (cteId, cte name, projection, data type, nullable) and builds a single Apply for a
representative, letting the other occurrences reuse its output. Correlated subqueries,
subqueries needing type coercion, DISTINCT/multi-column projections and non-CTE
subqueries are excluded to preserve semantics.

Release note

None

Check List (For Author)

  • Test

    • Unit Test
    • Regression test
    • Manual test
    • No need to test or manual test.
  • Behavior changed:

    • No.
  • Does this need documentation?

    • No.

…cing the same CTE

commit 56598d9a7a81157c12b3226702d114d6cf70a694
Author: lijinghan.1029 <lijinghan.1029@jd.com>
Date:   Fri Aug 14 17:54:54 2026 +0800

    (nereids) Reuse identical uncorrelated scalar subqueries referencing the same CTE

    When a query contains many structurally-identical uncorrelated scalar
    subqueries that reference the same CTE (e.g. (SELECT dt FROM deal_dt)
    repeated hundreds of times), SubqueryToApply generated one LogicalApply
    per occurrence, blowing up the Nereids optimization search space and
    causing the planner to time out.

    This change groups such reusable scalar subqueries within the same
    LogicalFilter/LogicalProject by (cteId, cte name, projection, data type,
    nullable) and builds a single Apply for the representative, letting the
    other occurrences reuse its output. Correlated subqueries, subqueries
    needing type coercion, DISTINCT/multi-column projections and non-CTE
    subqueries are excluded to preserve semantics.
@hello-stephen

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants