Skip to content

[CALCITE-2901] Preserve scalar RexSubQuery type with copyType#5027

Open
Soumyadeep-github wants to merge 1 commit into
apache:mainfrom
Soumyadeep-github:calcite-2901
Open

[CALCITE-2901] Preserve scalar RexSubQuery type with copyType#5027
Soumyadeep-github wants to merge 1 commit into
apache:mainfrom
Soumyadeep-github:calcite-2901

Conversation

@Soumyadeep-github

Copy link
Copy Markdown

JIRA: CALCITE-2901

Changes Proposed

  • Scalar sub-queries now take the type of the one column inside the sub-query (copyType). We stopped forcing that type to always allow null after that.
  • The SQL checker and RECORD_TO_SCALAR use the same idea so types stay in sync end to end.
  • After sub-queries are turned into joins, some fields can be null from the join even when the checker said “not null”. The code adds casts in a few places so the plan matches what the checker expects (sub-query remove rule, sql-to-rel, one aggregate rule).
  • Tests were updated: core diff tests, decorrelator string plans, Quidem scalar / sub-query, validator/coercion/table-function checks, rel-to-SQL for one MySQL-style case, operator tests in testkit, and two TPC-H plan checks in plus.

Preserve scalar RexSubQuery type with copyType; use copyType for scalar
sub-query Rex/SQL types instead of forcing nullable. Cast scalar sub-query
Rex to the validated type in SqlToRelConverter. Cast after SubQueryRemoveRule
and AggregateMinMaxToLimitRule when types drift. Update tests (IQ, XML, JUnit).
@sonarqubecloud

Copy link
Copy Markdown

@mihaibudiu mihaibudiu left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Other optimizations will remove useless cast, and some builder methods will not insert useless casts, but maybe you can check if all the casts you insert are necessary. There are quite a few changes.

+ "SELECT NULL) END AS `$f0`\n"
+ "FROM `foodmart`.`product`) AS `t0` ON TRUE\n"
+ "WHERE `product`.`net_weight` > CAST(`t0`.`$f0` AS DOUBLE)";
+ "WHERE `product`.`net_weight` > CAST(CAST(`t0`.`$f0` AS SIGNED) AS DOUBLE)";

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this case necessary?
If not, can you avoid inserting it?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for taking the time to review my changes! Agreed that we should not rely on later cast cleanup. I’ll go through the changed casts and drop any that aren’t needed for correctness.

For the CAST(CAST($f0 AS …) AS DOUBLE) case in RelToSqlConverterTest: I’ll trace where each cast is introduced and try to simplify to a single cast (or align types earlier) if semantics stay the same; if two steps are still required I’ll note why in a short comment or reply here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants