Skip to content

[VL] Support format_number function - #12754

Open
yikf wants to merge 1 commit into
apache:mainfrom
yikf:support-format-number
Open

[VL] Support format_number function#12754
yikf wants to merge 1 commit into
apache:mainfrom
yikf:support-format-number

Conversation

@yikf

@yikf yikf commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

What changes are proposed in this pull request?

Velox already registers the sparksql format_number(number, decimalPlaces) function (thousands separators, fixed decimal digits, HALF_EVEN rounding), so wire it up on the Gluten side.

Unlike CAST(number AS VARCHAR), this adds thousands separators and fixed decimal places. Supports tinyint, smallint, integer, bigint, float, and double.

How was this patch tested?

New UTs.

Was this patch authored or co-authored using generative AI tooling?

Yes, Assisted-by: Claude:claude-opus-4-8

Copilot AI lite review requested due to automatic review settings August 12, 2026 02:17
@github-actions github-actions Bot added CORE works for Gluten Core VELOX CLICKHOUSE DOCS labels Aug 12, 2026
@github-actions

Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR wires Spark SQL format_number(number, decimalPlaces) through Gluten’s Substrait expression mapping so Velox can execute it natively where supported, and adds coverage plus documentation/restriction metadata.

Changes:

  • Add format_number to ExpressionNames and register FormatNumber in ExpressionMappings.
  • Document Velox support as partial (numeric primitives only; DecimalType excluded) and add a corresponding restriction entry for doc generation.
  • Add Velox backend validation tests for format_number, including expected Spark-side execution for DecimalType input.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated no comments.

Show a summary per file
File Description
shims/common/src/main/scala/org/apache/gluten/expression/ExpressionNames.scala Adds the FORMAT_NUMBER function name constant used across shims/mappings.
gluten-substrait/src/main/scala/org/apache/gluten/expression/ExpressionMappings.scala Maps Spark’s FormatNumber expression to Substrait function name format_number.
docs/velox-backend-scalar-function-support.md Updates function support table to reflect partial support + DecimalType restriction note.
backends-velox/src/test/scala/org/apache/gluten/functions/ScalarFunctionsValidateSuite.scala Adds UTs covering offload for supported numeric types and non-offload for DecimalType input.
backends-velox/src/main/scala/org/apache/gluten/expression/ExpressionRestrictions.scala Adds a FormatNumberRestrictions entry for doc/restriction reporting.
backends-clickhouse/src/main/scala/org/apache/gluten/utils/CHExpressionUtil.scala Blacklists format_number for ClickHouse backend to avoid incorrect pushdown.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Copilot AI review requested due to automatic review settings August 12, 2026 11:26
@github-actions

Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated no new comments.

Suppressed comments (1)

backends-velox/src/test/scala/org/apache/gluten/functions/ScalarFunctionsValidateSuite.scala:768

  • The added test comments say this block exercises HALF_EVEN rounding and thousands separators, but the current inputs from TPCH columns are not deterministic for those edge cases (e.g., they may not hit tie-to-even cases, and values may not exceed 999 to guarantee separators). Adding deterministic cases (using datatab.double_field1 which already includes 1.025/1.035/1.045, and a scaled integer) will make the intent of the test reliably true.
    // Floating-point input, exercising HALF_EVEN rounding and thousands separators.
    runQueryAndCompare(
      "SELECT format_number(cast(l_quantity as double), 1) FROM lineitem limit 50") {
      checkGlutenPlan[ProjectExecTransformer]
    }

@yikf
yikf force-pushed the support-format-number branch from 20b7e3d to ba2e2b4 Compare August 12, 2026 11:39
@github-actions

Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants