[SPARK-59063][SQL] Use a byte-length guard in LikeSimplification for 'prefix%suffix' - #58362
Open
david-mollitor-db wants to merge 1 commit into
Open
Conversation
…'prefix%suffix'
`LikeSimplification` rewrites `col LIKE 'prefix%suffix'` into
Length(col) >= numChars(prefix) + numChars(suffix)
&& StartsWith(col, prefix) && EndsWith(col, suffix)
The length guard exists only to reject strings too short to hold both the
prefix and the suffix (e.g. 'a' must not match 'a%a'). `Length` is
`numChars`, an O(N) code-point scan of the string.
Because `StartsWith` and `EndsWith` already pin the prefix and suffix at
code-point boundaries, a byte-length floor accepts exactly the same strings
as the code-point floor: any string long enough in bytes to contain both
anchored substrings is also long enough in code points, and vice versa.
So the guard can use `OctetLength` (the stored `numBytes`, O(1)) with the
threshold expressed in bytes, avoiding the per-row character-count scan in
the residual filter while preserving behavior.
Updated `LikeSimplificationSuite` accordingly. For ASCII prefixes/suffixes
the byte threshold equals the old code-point threshold; the emoji case now
guards on 8 bytes instead of 2 code points (each of the two 4-byte code
points), which is the same set of accepted strings.
Generated-by: Claude Opus 4.8
david-mollitor-db
force-pushed
the
octet-length-like-simplification
branch
from
August 27, 2026 21:30
b24f682 to
553d461
Compare
david-mollitor-db
changed the title
david-mollitor-db
marked this pull request as ready for review
August 27, 2026 21:30
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
LikeSimplificationrewritescol LIKE 'prefix%suffix'into a length guard plusStartsWith/EndsWith:This PR changes the length guard to use
OctetLength(byte length) instead ofLength(character length), with the threshold expressed in bytes:
The guard exists only to reject strings too short to hold both the prefix and the suffix
(e.g.
'a'must not match'a%a').LengthisnumChars, which is an O(N) code-pointscan of the input string;
OctetLengthis the storednumBytes, which is O(1).Why are the changes needed?
The character-length guard runs per row in the residual filter and performs an O(N)
code-point count, whereas the byte length is already stored on
UTF8Stringand is O(1).The two guards accept exactly the same set of strings, so the swap is behavior-preserving.
StartsWithandEndsWithalready pin the prefix and suffix at code-point boundaries, soany string that satisfies both anchors and is long enough in bytes to contain them is also
long enough in code points, and vice versa. Concretely, for a string that already passes
StartsWith(prefix) && EndsWith(suffix):Does this PR introduce any user-facing change?
No. The rewrite accepts the same rows as before; only the internal guard expression changes
(
length->octet_length).How was this patch tested?
Updated and re-ran
LikeSimplificationSuite(18/18 passing). For ASCII prefixes/suffixesthe byte threshold equals the previous code-point threshold (
length(a) >= 6becomesoctet_length(a) >= 6). The emoji case now guards on 8 bytes instead of 2 code points --each of the two code points is a 4-byte UTF-8 sequence -- which accepts the same strings.
Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Opus 4.8