BE-614: HashQL: Generate synthetic closure bodies for intrinsics used as first-class values#8895
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
2 Skipped Deployments
|
PR SummaryMedium Risk Overview Reification now routes qualified variables through Small supporting changes: Reviewed by Cursor Bugbot for commit 0c451c1. Bugbot is set up for automated code reviews on this repo. Configure here. |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## bm/be-615-hashql-introduce-synthetic-closures-and-trivial-closure #8895 +/- ##
======================================================================================================
- Coverage 31.11% 19.37% -11.74%
======================================================================================================
Files 589 559 -30
Lines 31808 26498 -5310
Branches 3326 3139 -187
======================================================================================================
- Hits 9896 5135 -4761
+ Misses 21769 21265 -504
+ Partials 143 98 -45
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Merging this PR will improve performance by 38.94%
Warning Please fix the performance issues or acknowledge them on CodSpeed. Performance Changes
Tip Investigate this regression by commenting Comparing |
410fa1b to
ac8ed9e
Compare
538afac to
88472a7
Compare
| /// Constructs a `&'static [ConstantSymbol]` path from `::` separated segments. | ||
| /// | ||
| /// Each segment maps to the corresponding `sym::` constant. | ||
| /// | ||
| /// ```ignore | ||
| /// T![::core::math::add] // expands to &[sym::core, sym::math, sym::add] | ||
| /// T![::core::cmp::eq] // expands to &[sym::core, sym::cmp, sym::eq] | ||
| /// ``` |
88472a7 to
d3b03b8
Compare
ac8ed9e to
4aacfd8
Compare
4aacfd8 to
f2b45cb
Compare
d3b03b8 to
015b737
Compare
f2b45cb to
489dd1a
Compare
015b737 to
0694d8d
Compare
489dd1a to
5628792
Compare
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 4580488. Configure here.
| /// Constructs a `&'static [ConstantSymbol]` path from `::` separated segments. | ||
| /// | ||
| /// Each segment maps to the corresponding `sym::` constant. | ||
| /// | ||
| /// ```ignore | ||
| /// T![::core::math::add] // expands to &[sym::core, sym::math, sym::add] | ||
| /// T![::core::cmp::eq] // expands to &[sym::core, sym::cmp, sym::eq] | ||
| /// ``` |
| /// ```ignore | ||
| /// T![::core::math::add] // expands to &[sym::core, sym::math, sym::add] | ||
| /// T![::core::cmp::eq] // expands to &[sym::core, sym::cmp, sym::eq] | ||
| /// ``` |
Benchmark results
|
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| resolve_policies_for_actor | user: empty, selectivity: high, policies: 2002 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: medium, policies: 1002 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: high, policies: 3314 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: medium, policies: 1527 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: high, policies: 2078 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: medium, policies: 1033 | Flame Graph |
policy_resolution_medium
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| resolve_policies_for_actor | user: empty, selectivity: high, policies: 102 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: medium, policies: 52 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: high, policies: 269 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: medium, policies: 108 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: high, policies: 133 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: medium, policies: 63 | Flame Graph |
policy_resolution_none
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| resolve_policies_for_actor | user: empty, selectivity: high, policies: 2 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: medium, policies: 2 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: high, policies: 8 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: medium, policies: 3 | Flame Graph |
policy_resolution_small
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| resolve_policies_for_actor | user: empty, selectivity: high, policies: 52 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: medium, policies: 26 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: high, policies: 94 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: medium, policies: 27 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: high, policies: 66 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: medium, policies: 29 | Flame Graph |
read_scaling_complete
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| entity_by_id;one_depth | 1 entities | Flame Graph | |
| entity_by_id;one_depth | 10 entities | Flame Graph | |
| entity_by_id;one_depth | 25 entities | Flame Graph | |
| entity_by_id;one_depth | 5 entities | Flame Graph | |
| entity_by_id;one_depth | 50 entities | Flame Graph | |
| entity_by_id;two_depth | 1 entities | Flame Graph | |
| entity_by_id;two_depth | 10 entities | Flame Graph | |
| entity_by_id;two_depth | 25 entities | Flame Graph | |
| entity_by_id;two_depth | 5 entities | Flame Graph | |
| entity_by_id;two_depth | 50 entities | Flame Graph | |
| entity_by_id;zero_depth | 1 entities | Flame Graph | |
| entity_by_id;zero_depth | 10 entities | Flame Graph | |
| entity_by_id;zero_depth | 25 entities | Flame Graph | |
| entity_by_id;zero_depth | 5 entities | Flame Graph | |
| entity_by_id;zero_depth | 50 entities | Flame Graph |
read_scaling_linkless
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| entity_by_id | 1 entities | Flame Graph | |
| entity_by_id | 10 entities | Flame Graph | |
| entity_by_id | 100 entities | Flame Graph | |
| entity_by_id | 1000 entities | Flame Graph | |
| entity_by_id | 10000 entities | Flame Graph |
representative_read_entity
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/block/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/book/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/building/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/organization/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/page/v/2
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/person/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/playlist/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/song/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/uk-address/v/1
|
Flame Graph |
representative_read_entity_type
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| get_entity_type_by_id | Account ID: bf5a9ef5-dc3b-43cf-a291-6210c0321eba
|
Flame Graph |
representative_read_multiple_entities
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| entity_by_property | traversal_paths=0 | 0 | |
| entity_by_property | traversal_paths=255 | 1,resolve_depths=inherit:1;values:255;properties:255;links:127;link_dests:126;type:true | |
| entity_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:0;links:0;link_dests:0;type:false | |
| entity_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:0;links:1;link_dests:0;type:true | |
| entity_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:2;links:1;link_dests:0;type:true | |
| entity_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:2;properties:2;links:1;link_dests:0;type:true | |
| link_by_source_by_property | traversal_paths=0 | 0 | |
| link_by_source_by_property | traversal_paths=255 | 1,resolve_depths=inherit:1;values:255;properties:255;links:127;link_dests:126;type:true | |
| link_by_source_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:0;links:0;link_dests:0;type:false | |
| link_by_source_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:0;links:1;link_dests:0;type:true | |
| link_by_source_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:2;links:1;link_dests:0;type:true | |
| link_by_source_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:2;properties:2;links:1;link_dests:0;type:true |
scenarios
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| full_test | query-limited | Flame Graph | |
| full_test | query-unlimited | Flame Graph | |
| linked_queries | query-limited | Flame Graph | |
| linked_queries | query-unlimited | Flame Graph |


🌟 What is the purpose of this PR?
Allows intrinsics to be used in value position (passed as arguments, bound to variables) by generating synthetic MIR wrapper bodies during reification. Previously, intrinsics like
<=could only appear at call sites where specialization rewrites them. Now they can flow through higher-order functions:🔍 What does this change?
Synthetic body generation:
Syntheticsstruct inCrossCompileStatemanages creation and caching of wrapper bodiesSyntheticBuildergenerates MIR bodies with fat closure ABI (unit env as first parameter)T!macro for matching qualified paths againstConstantSymbolarrays:&T![::core::cmp::gt]in slice patterns,sym::path::core::cmp::gt::CONSTfor the namebinary!/unary!macros deriving path and name from the same segmentsThin-call specialization:
rvalue_call_thin_specializerecognizesCall(Thin, Qualified(intrinsic), [])(the administrative thunk-force from the thunking phase) and produces the closure aggregate directly, skipping thunk body generation() -> ClosureTypewrapper to extract the actual signatureIntrinsic classification:
gt,lt,gte,lte,eq,ne), boolean (and,or,not), arithmetic (add,sub), bitwise (and,or,not)!payload):mul,div,rem,mod,pow,xor,shl,shrsqrt,cbrt,rootgraph::head::entities,graph::body::filter,graph::tail::collectproduce a specific user-facing diagnosticDiagnostics:
intrinsic_not_first_class: user-facing error for graph intrinsics in value positionsynthetic_binary_arity_mismatch/synthetic_unary_arity_mismatch: ICE for monomorphized type invariant violationsPre-Merge Checklist 🚀
🚢 Has this modified a publishable library?
This PR:
📜 Does this require a change to the docs?
The changes in this PR:
🕸️ Does this require a change to the Turbo Graph?
The changes in this PR:
"<="as the entire program) is not supported. The thunking phase does not wrap the root body when it is a variable, so it never reaches the synthetic machinery. Tracked with arun: skiptest.🐾 Next steps
🛡 What tests cover this?
Compiletests (8 new):
reify/synthetic-intrinsic-value: basic synthetic body generation and thin-call specializationreify/synthetic-intrinsic-reused: same intrinsic used twice shares one body (caching)reify/synthetic-intrinsic-multiple: different intrinsics get separate bodiesreify/synthetic-graph-not-first-class: graph intrinsic in value position produces errorreify/synthetic-intrinsic-bare: bare intrinsic at root (run: skip)interpret/synthetic-intrinsic-value: end-to-end correctness (evaluates totrue)post_inline/synthetic-intrinsic-hof: full optimization cascade with constants (collapses toreturn true)post_inline/synthetic-intrinsic-hof-dynamic: HOF with dynamic inputs collapses to barex <= y❓ How to test this?
cargo run --package hashql-compiletest -- run --filter "test(synthetic)"📹 Demo
The
post_inline/synthetic-intrinsic-hof-dynamictest shows the full cascade. A higher-order function receiving<=as a value with dynamic inputs:All indirection eliminated. Indistinguishable from writing
["<=", x, y]directly.