BE-629: Add spherical k-means entity clustering endpoint via `/entities/embeddings/clusters` by indietyp · Pull Request #8919 · hashintel/hash

indietyp · 2026-06-30T09:46:00Z

🌟 What is the purpose of this PR?

This PR adds a POST /entities/embeddings/clusters endpoint that groups a set of entities by embedding similarity using spherical k-means clustering. Callers supply a list of entity IDs, a desired cluster count, and an optional embedding dimension (matryoshka truncation). The response contains the cluster assignments with unit-normalized centroids, plus a list of entities that had no stored embedding.

The clustering algorithm is implemented from scratch in Rust using SIMD-accelerated kernels (f32x8), k-means++ seeding, multiple restarts, and parallel assignment via Rayon. Embeddings are truncated server-side in Postgres using subvector before being sent over the wire, keeping network cost proportional to the requested dimension rather than the full stored width.

The implementation is up to 48x faster than existing crates that operate on CPUs.

🔍 What does this change?

Adds a Dimension newtype that enforces the positive-multiple-of-8 invariant required by the SIMD kernels.
Adds a kernel module with SIMD-accelerated primitives: dot, add_into, scale_into, scale, add_scaled_into, normalize, micro_4x2 (4-point × 2-centroid tiled dot product), and nearest4 (nearest-centroid search for 4 points simultaneously).
Adds a clustering module implementing spherical k-means with k-means++ D² seeding, Lloyd iterations, empty-cluster reseeding, convergence tolerance, and configurable restarts via a Config struct.
Adds ClusterEntitiesParams, EntityCluster, and ClusterEntitiesResponse types to the entity store API.
Adds a ClusterError error type covering invalid dimension, dimension-too-large, and store failure cases.
Adds cluster_entities to the EntityStore trait and implements it in the Postgres store, including permission filtering that avoids leaking which entity IDs were denied versus missing embeddings.
Registers the new endpoint at POST /entities/embeddings/clusters and nests the existing POST /entities/embeddings handler under /entities/embeddings/ to keep the routing consistent.
Exposes the new types and endpoint in the OpenAPI schema.
Forwards cluster_entities through the type-fetcher store wrapper and the integration test DatabaseApi shim.

Pre-Merge Checklist 🚀

🚢 Has this modified a publishable library?

This PR:

does not modify any publishable blocks or libraries, or modifications do not need publishing

📜 Does this require a change to the docs?

The changes in this PR:

are internal and do not require a docs change

🕸️ Does this require a change to the Turbo Graph?

The changes in this PR:

do not affect the execution graph

🛡 What tests cover this?

Unit tests for squared_chord_distance covering identical, orthogonal, opposite, and zero-norm cases.
Unit tests for all SIMD kernel functions (dot, add_into, scale_into, scale, add_scaled_into, normalize, micro_4x2, nearest4) verified against scalar reference implementations.
Unit tests for the clustering algorithm covering: empty input, k=0, k=1, single point, n < 4, n = k, well-separated blobs (>95% accuracy), determinism with the same seed, unit-normalized centroids, labels in range, labels nearest to assigned centroid, subsampled clustering, more clusters than natural groups, all-identical points, and mixed zero-norm rows.
Unit tests for the Dimension newtype covering valid multiples of 8, zero rejection, and non-multiples rejection.

❓ How to test this?

Checkout the branch.
Ensure entities with stored embeddings exist in the database.
Send a POST /entities/embeddings/clusters request with a JSON body containing entityIds, clusterCount, and optionally dimension and seed.
Confirm the response contains clusters (each with clusterId, entityIds, and centroid) and missingEmbeddings for any entities without stored embeddings.

feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: embedding clustering feat: embedding clustering feat: embedding clustering feat: embedding clustering feat: checkpoint feat: checkpoint feat: checkpoint fix: merge feat: checkpoint feat: checkpoint feat: checkpoint fix: merge feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint] feat: checkpoint] feat: checkpoint] feat: checkpoint feat: checkpoint

vercel · 2026-06-30T09:46:05Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
hash	Ready	Preview, Comment	Jul 7, 2026 7:46pm
petrinaut	Ready	Preview	Jul 7, 2026 7:46pm

1 Skipped Deployment

Project	Deployment	Actions	Updated (UTC)
hashdotdesign-tokens	Ignored	Preview	Jul 7, 2026 7:46pm

cursor · 2026-06-30T09:46:09Z

PR Summary

Medium Risk
New authenticated API and CPU-heavy clustering on request paths; permission handling is deliberate but the endpoint adds load and new failure modes (panics bridged from Rayon, dimension/k limits).

Overview
Adds POST /entities/embeddings/clusters so callers can group entities by stored embedding similarity. The request takes entity IDs, cluster count, optional matryoshka dimension (default 256, max 512) and seed; the response returns clusters with centroids and inertia, plus missingEmbeddings for IDs without embeddings (permission denials are not distinguished from missing data).

hash-graph-embeddings gains a new spherical k-means stack: SIMD f32x8 kernels, Dimension (multiple-of-8), k-means++ seeding, Lloyd iterations with Rayon parallelism, and CodSpeed benchmarks. The Postgres store loads truncated vectors via subvector, enforces ViewEntity permissions, and runs clustering on a global Rayon pool (initialized once in hash-graph with half of available cores). Embedding update routing moves under /entities/embeddings/ with clusters at /clusters.

Store/API types (ClusterEntitiesParams, ClusterError, etc.), OpenAPI, type-fetcher forwarding, and CI warm-up for CodSpeed are included.

^{Reviewed by Cursor Bugbot for commit 795ea71. Bugbot is set up for automated code reviews on this repo. Configure here.}

indietyp · 2026-06-30T09:46:21Z

This stack of pull requests is managed by Graphite. Learn more about stacking.

Copilot

Pull request overview

Adds embedding-based spherical k-means clustering to the graph store and exposes it via a new REST endpoint (POST /entities/embeddings/clusters). The implementation introduces a SIMD-accelerated Rust clustering engine and wires it through the Postgres store, API routing/OpenAPI, and relevant store wrappers/shims.

Changes:

Introduces a new embedding module in hash_graph_store with a Dimension invariant type, SIMD kernels, and a spherical k-means implementation.
Extends the EntityStore API with cluster_entities and implements it in the Postgres store, including permission filtering and embedding truncation via subvector.
Adds the REST endpoint and forwards the new store method through type-fetcher and integration test shims.

Reviewed changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
tests/graph/integration/postgres/lib.rs	Forwards `cluster_entities` through the `DatabaseApi` integration shim.
libs/@local/graph/type-fetcher/src/store.rs	Forwards `cluster_entities` through the type-fetcher store wrapper.
libs/@local/graph/store/src/lib.rs	Enables required nightly features and registers the new `embedding` module.
libs/@local/graph/store/src/error.rs	Adds `ClusterError` for clustering-related failures.
libs/@local/graph/store/src/entity/store.rs	Adds request/response types and the `EntityStore::cluster_entities` trait method.
libs/@local/graph/store/src/entity/mod.rs	Re-exports the new clustering API types.
libs/@local/graph/store/src/embedding/mod.rs	Declares the new embedding submodules and lint expectations.
libs/@local/graph/store/src/embedding/kernel.rs	Implements SIMD-accelerated vector primitives and tests.
libs/@local/graph/store/src/embedding/dimension.rs	Adds `Dimension` newtype enforcing “positive multiple of 8”.
libs/@local/graph/store/src/embedding/clustering.rs	Implements spherical k-means (+ seeding/restarts/parallel assignment) and tests.
libs/@local/graph/postgres-store/src/store/postgres/knowledge/entity/mod.rs	Implements `cluster_entities` query + permission filtering + clustering execution.
libs/@local/graph/api/src/rest/entity.rs	Registers the new REST endpoint and nests existing embeddings routing.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

codecov · 2026-07-01T08:11:30Z

Codecov Report

❌ Patch coverage is 92.58786% with 116 lines in your changes missing coverage. Please review.
✅ Project coverage is 60.94%. Comparing base (186b284) to head (795ea71).

Files with missing lines	Patch %	Lines
...s-store/src/store/postgres/knowledge/entity/mod.rs	0.00%	59 Missing ⚠️
libs/@local/graph/api/src/rest/entity/mod.rs	0.00%	24 Missing ⚠️
libs/@local/graph/embeddings/src/clustering.rs	98.47%	11 Missing and 4 partials ⚠️
libs/@local/graph/embeddings/src/kernel.rs	98.12%	4 Missing and 5 partials ⚠️
libs/@local/graph/embeddings/src/dimension.rs	58.82%	5 Missing and 2 partials ⚠️
libs/@local/graph/store/src/entity/store.rs	0.00%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #8919      +/-   ##
==========================================
+ Coverage   59.20%   60.94%   +1.74%     
==========================================
  Files        1358     1197     -161     
  Lines      132674   126137    -6537     
  Branches     6019     5231     -788     
==========================================
- Hits        78546    76875    -1671     
+ Misses      53198    48381    -4817     
+ Partials      930      881      -49

Flag	Coverage Δ
apps.hash-ai-worker-py	`?`
apps.hash-ai-worker-ts	`1.39% <ø> (ø)`
apps.hash-api	`6.39% <ø> (ø)`
backend-integration-tests	`?`
blockprotocol.type-system	`40.84% <ø> (ø)`
deer	`?`
error-stack	`?`
local.claude-hooks	`0.00% <ø> (ø)`
local.harpc-client	`51.49% <ø> (ø)`
local.hash-backend-utils	`?`
local.hash-graph-sdk	`10.02% <ø> (ø)`
local.hash-isomorphic-utils	`0.18% <ø> (ø)`
local.hash-subgraph	`?`
rust.antsi	`?`
rust.deer	`?`
rust.error-stack	`90.89% <ø> (ø)`
rust.harpc-codec	`84.70% <ø> (ø)`
rust.harpc-net	`96.21% <ø> (+0.01%)`	⬆️
rust.harpc-tower	`67.03% <ø> (ø)`
rust.harpc-types	`0.00% <ø> (ø)`
rust.harpc-wire-protocol	`92.23% <ø> (ø)`
rust.hash-codec	`72.76% <ø> (ø)`
rust.hash-graph-api	`7.43% <0.00%> (-0.04%)`	⬇️
rust.hash-graph-authorization	`62.59% <ø> (ø)`
rust.hash-graph-embeddings	`91.88% <97.90%> (?)`
rust.hash-graph-postgres-store	`29.02% <0.00%> (-0.10%)`	⬇️
rust.hash-graph-store	`38.20% <0.00%> (-0.02%)`	⬇️
rust.hash-graph-temporal-versioning	`47.95% <ø> (ø)`
rust.hash-graph-types	`0.00% <ø> (ø)`
rust.hash-graph-validation	`83.43% <ø> (ø)`
rust.hashql-ast	`89.63% <ø> (+2.39%)`	⬆️
rust.hashql-compiletest	`28.39% <ø> (ø)`
rust.hashql-core	`78.95% <ø> (-0.47%)`	⬇️
rust.hashql-diagnostics	`72.51% <ø> (+0.19%)`	⬆️
rust.hashql-eval	`79.47% <ø> (ø)`
rust.hashql-hir	`89.09% <ø> (+0.02%)`	⬆️
rust.hashql-mir	`88.05% <ø> (+<0.01%)`	⬆️
rust.hashql-syntax-jexpr	`94.04% <ø> (ø)`
rust.sarif	`?`
sarif	`?`
tests.hash-backend-integration	`?`
unit-tests	`?`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

codspeed-hq · 2026-07-01T08:15:09Z

Merging this PR will degrade performance by 15.38%

❌ 2 regressed benchmarks
✅ 78 untouched benchmarks
🆕 18 new benchmarks

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Benchmark	`BASE`	`HEAD`	Efficiency
❌	`bit_matrix/dense/iter_row[64]`	140.8 ns	170 ns	-17.16%
❌	`bit_matrix/dense/iter_row[200]`	185.8 ns	215 ns	-13.57%
🆕	`n10000_d256[128]`	N/A	4 s	N/A
🆕	`n10000_d256[32]`	N/A	1 s	N/A
🆕	`n10000_d256[8]`	N/A	94.3 ms	N/A
🆕	`n50000_d256[32]`	N/A	1.5 s	N/A
🆕	`kernel/add_scaled_into[1536]`	N/A	6.9 µs	N/A
🆕	`kernel/add_scaled_into[256]`	N/A	1.4 µs	N/A
🆕	`kernel/add_scaled_into[3072]`	N/A	13.4 µs	N/A
🆕	`kernel/dot[1536]`	N/A	6.6 µs	N/A
🆕	`kernel/dot[256]`	N/A	1.3 µs	N/A
🆕	`kernel/dot[3072]`	N/A	12.9 µs	N/A
🆕	`kernel/micro_4x2[1536]`	N/A	23.9 µs	N/A
🆕	`kernel/micro_4x2[256]`	N/A	4.9 µs	N/A
🆕	`kernel/micro_4x2[3072]`	N/A	46.7 µs	N/A
🆕	`d1536[16]`	N/A	111.6 µs	N/A
🆕	`d256[15]`	N/A	18.6 µs	N/A
🆕	`d256[16]`	N/A	19.5 µs	N/A
🆕	`d256[64]`	N/A	68.3 µs	N/A
🆕	`d3072[16]`	N/A	221.2 µs	N/A

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.

_{Comparing bm/be-629-implement-kmeans-clustering-in-the-hash-graph (795ea71) with main (f52f62c)¹}

No successful run was found on main (186b284) during the generation of this report, so f52f62c was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

Copilot

Pull request overview

Copilot reviewed 15 out of 16 changed files in this pull request and generated 3 comments.

Copilot

Pull request overview

Copilot reviewed 38 out of 40 changed files in this pull request and generated 4 comments.

Copilot

Pull request overview

Copilot reviewed 43 out of 45 changed files in this pull request and generated 5 comments.

Copilot

Pull request overview

Copilot reviewed 43 out of 45 changed files in this pull request and generated 3 comments.

+async fn cluster_entities<S>(
+    AuthenticatedUserHeader(actor_id): AuthenticatedUserHeader,
+    store_pool: Extension<Arc<S>>,
+    temporal_client: Extension<Option<Arc<TemporalClient>>>,
+    Json(params): Json<ClusterEntitiesParams>,


cursor

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 795ea71. Configure here.}

cursor · 2026-07-07T19:34:25Z

+                    FROM entity_embeddings e
+                    WHERE e.property IS NULL
+                      AND (e.web_id, e.entity_uuid) IN (SELECT * FROM unnest($1::uuid[], \
+                     $2::uuid[]))"


Embedding query ignores draft scope

Medium Severity

cluster_entities checks view permission with include_drafts: false, but the embedding SELECT does not restrict rows to draft_id IS NULL. Unlike the delete/upsert paths in the same module, clustering can load draft-scoped embedding rows for callers that only have access to published entities.

^{Reviewed by Cursor Bugbot for commit 795ea71. Configure here.}

Copilot

Pull request overview

Copilot reviewed 46 out of 48 changed files in this pull request and generated 3 comments.

+                    FROM entity_embeddings e
+                    WHERE e.property IS NULL
+                      AND (e.web_id, e.entity_uuid) IN (SELECT * FROM unnest($1::uuid[], \
+                     $2::uuid[]))"


+        let found_set: HashSet<(WebId, EntityUuid)> = found_ids
+            .iter()
+            .map(|id| (id.web_id, id.entity_uuid))
+            .collect();
+        let missing_embeddings: Vec<EntityId> = params
+            .entity_ids
+            .into_iter()
+            .filter(|id| !found_set.contains(&(id.web_id, id.entity_uuid)))
+            .collect();


+    responses(
+        (status = 200, content_type = "application/json", description = "Clusters of entities by embedding similarity", body = ClusterEntitiesResponse),
+        (status = 422, content_type = "text/plain", description = "Provided request body is invalid"),
+
+        (status = 500, description = "Store error occurred"),
+    ),


github-actions · 2026-07-07T20:46:28Z

Benchmark results

@rust/hash-graph-benches – Integrations

policy_resolution_large

Function	Value	Mean	Flame graphs
resolve_policies_for_actor	user: empty, selectivity: high, policies: 2002	$$20.6 \mathrm{ms} \pm 148 \mathrm{μs}\left({\color{gray}-3.096 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: empty, selectivity: low, policies: 1	$$2.70 \mathrm{ms} \pm 17.4 \mathrm{μs}\left({\color{gray}-3.534 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: empty, selectivity: medium, policies: 1002	$$10.6 \mathrm{ms} \pm 69.6 \mathrm{μs}\left({\color{lightgreen}-7.952 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: seeded, selectivity: high, policies: 3314	$$34.6 \mathrm{ms} \pm 293 \mathrm{μs}\left({\color{gray}-2.771 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: seeded, selectivity: low, policies: 1	$$12.8 \mathrm{ms} \pm 115 \mathrm{μs}\left({\color{gray}-2.511 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: seeded, selectivity: medium, policies: 1527	$$20.4 \mathrm{ms} \pm 146 \mathrm{μs}\left({\color{gray}-0.272 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: system, selectivity: high, policies: 2078	$$21.2 \mathrm{ms} \pm 144 \mathrm{μs}\left({\color{gray}-3.931 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: system, selectivity: low, policies: 1	$$3.00 \mathrm{ms} \pm 16.1 \mathrm{μs}\left({\color{gray}-4.802 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: system, selectivity: medium, policies: 1033	$$11.6 \mathrm{ms} \pm 84.0 \mathrm{μs}\left({\color{lightgreen}-6.824 \mathrm{\%}}\right) $$	Flame Graph

policy_resolution_medium

Function	Value	Mean	Flame graphs
resolve_policies_for_actor	user: empty, selectivity: high, policies: 102	$$2.95 \mathrm{ms} \pm 21.2 \mathrm{μs}\left({\color{gray}-1.001 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: empty, selectivity: low, policies: 1	$$2.37 \mathrm{ms} \pm 15.0 \mathrm{μs}\left({\color{gray}-4.910 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: empty, selectivity: medium, policies: 52	$$2.65 \mathrm{ms} \pm 13.4 \mathrm{μs}\left({\color{gray}-3.583 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: seeded, selectivity: high, policies: 269	$$4.05 \mathrm{ms} \pm 22.0 \mathrm{μs}\left({\color{gray}-4.992 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: seeded, selectivity: low, policies: 1	$$2.83 \mathrm{ms} \pm 19.7 \mathrm{μs}\left({\color{gray}-1.065 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: seeded, selectivity: medium, policies: 108	$$3.22 \mathrm{ms} \pm 17.6 \mathrm{μs}\left({\color{gray}-0.650 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: system, selectivity: high, policies: 133	$$3.41 \mathrm{ms} \pm 25.3 \mathrm{μs}\left({\color{gray}-1.061 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: system, selectivity: low, policies: 1	$$2.71 \mathrm{ms} \pm 13.1 \mathrm{μs}\left({\color{gray}-2.615 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: system, selectivity: medium, policies: 63	$$3.16 \mathrm{ms} \pm 16.6 \mathrm{μs}\left({\color{lightgreen}-5.896 \mathrm{\%}}\right) $$	Flame Graph

policy_resolution_none

Function	Value	Mean	Flame graphs
resolve_policies_for_actor	user: empty, selectivity: high, policies: 2	$$2.11 \mathrm{ms} \pm 10.9 \mathrm{μs}\left({\color{gray}-1.018 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: empty, selectivity: low, policies: 1	$$1.97 \mathrm{ms} \pm 9.96 \mathrm{μs}\left({\color{gray}-0.594 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: empty, selectivity: medium, policies: 2	$$2.07 \mathrm{ms} \pm 7.61 \mathrm{μs}\left({\color{gray}-1.732 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: system, selectivity: high, policies: 8	$$2.34 \mathrm{ms} \pm 14.9 \mathrm{μs}\left({\color{gray}-0.747 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: system, selectivity: low, policies: 1	$$2.16 \mathrm{ms} \pm 17.6 \mathrm{μs}\left({\color{gray}-0.159 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: system, selectivity: medium, policies: 3	$$2.29 \mathrm{ms} \pm 12.8 \mathrm{μs}\left({\color{gray}-1.398 \mathrm{\%}}\right) $$	Flame Graph

policy_resolution_small

Function	Value	Mean	Flame graphs
resolve_policies_for_actor	user: empty, selectivity: high, policies: 52	$$2.39 \mathrm{ms} \pm 13.4 \mathrm{μs}\left({\color{gray}-2.206 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: empty, selectivity: low, policies: 1	$$2.17 \mathrm{ms} \pm 11.1 \mathrm{μs}\left({\color{gray}-0.158 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: empty, selectivity: medium, policies: 26	$$2.32 \mathrm{ms} \pm 15.6 \mathrm{μs}\left({\color{gray}-2.744 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: seeded, selectivity: high, policies: 94	$$2.65 \mathrm{ms} \pm 15.0 \mathrm{μs}\left({\color{gray}-1.676 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: seeded, selectivity: low, policies: 1	$$2.36 \mathrm{ms} \pm 10.0 \mathrm{μs}\left({\color{gray}-1.480 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: seeded, selectivity: medium, policies: 27	$$2.56 \mathrm{ms} \pm 17.8 \mathrm{μs}\left({\color{gray}-2.047 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: system, selectivity: high, policies: 66	$$2.62 \mathrm{ms} \pm 14.7 \mathrm{μs}\left({\color{gray}-0.528 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: system, selectivity: low, policies: 1	$$2.32 \mathrm{ms} \pm 9.20 \mathrm{μs}\left({\color{gray}-2.493 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: system, selectivity: medium, policies: 29	$$2.54 \mathrm{ms} \pm 14.2 \mathrm{μs}\left({\color{gray}-3.277 \mathrm{\%}}\right) $$	Flame Graph

read_scaling_complete

Function	Value	Mean	Flame graphs
entity_by_id;one_depth	1 entities	$$33.3 \mathrm{ms} \pm 185 \mathrm{μs}\left({\color{gray}0.338 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id;one_depth	10 entities	$$26.5 \mathrm{ms} \pm 147 \mathrm{μs}\left({\color{gray}1.05 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id;one_depth	25 entities	$$28.4 \mathrm{ms} \pm 153 \mathrm{μs}\left({\color{gray}-1.630 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id;one_depth	5 entities	$$25.0 \mathrm{ms} \pm 132 \mathrm{μs}\left({\color{gray}0.267 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id;one_depth	50 entities	$$34.7 \mathrm{ms} \pm 223 \mathrm{μs}\left({\color{gray}-2.153 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id;two_depth	1 entities	$$38.4 \mathrm{ms} \pm 219 \mathrm{μs}\left({\color{gray}-4.737 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id;two_depth	10 entities	$$33.4 \mathrm{ms} \pm 223 \mathrm{μs}\left({\color{gray}4.51 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id;two_depth	25 entities	$$74.8 \mathrm{ms} \pm 477 \mathrm{μs}\left({\color{gray}-0.425 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id;two_depth	5 entities	$$27.8 \mathrm{ms} \pm 184 \mathrm{μs}\left({\color{gray}-0.339 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id;two_depth	50 entities	$$246 \mathrm{ms} \pm 1.13 \mathrm{ms}\left({\color{red}10.2 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id;zero_depth	1 entities	$$8.42 \mathrm{ms} \pm 70.1 \mathrm{μs}\left({\color{gray}-2.574 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id;zero_depth	10 entities	$$8.54 \mathrm{ms} \pm 52.0 \mathrm{μs}\left({\color{gray}-4.260 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id;zero_depth	25 entities	$$8.42 \mathrm{ms} \pm 45.4 \mathrm{μs}\left({\color{lightgreen}-5.276 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id;zero_depth	5 entities	$$8.39 \mathrm{ms} \pm 43.2 \mathrm{μs}\left({\color{gray}-4.225 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id;zero_depth	50 entities	$$8.44 \mathrm{ms} \pm 37.2 \mathrm{μs}\left({\color{lightgreen}-9.070 \mathrm{\%}}\right) $$	Flame Graph

read_scaling_linkless

Function	Value	Mean	Flame graphs
entity_by_id	1 entities	$$8.57 \mathrm{ms} \pm 61.2 \mathrm{μs}\left({\color{lightgreen}-6.392 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id	10 entities	$$8.52 \mathrm{ms} \pm 49.4 \mathrm{μs}\left({\color{lightgreen}-5.055 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id	100 entities	$$8.42 \mathrm{ms} \pm 42.9 \mathrm{μs}\left({\color{lightgreen}-5.462 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id	1000 entities	$$8.89 \mathrm{ms} \pm 63.2 \mathrm{μs}\left({\color{gray}-2.627 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id	10000 entities	$$8.90 \mathrm{ms} \pm 45.7 \mathrm{μs}\left({\color{lightgreen}-7.695 \mathrm{\%}}\right) $$	Flame Graph

representative_read_entity

Function	Value	Mean	Flame graphs
entity_by_id	entity type ID: `https://blockprotocol.org/@alice/types/entity-type/block/v/1`	$$8.80 \mathrm{ms} \pm 42.9 \mathrm{μs}\left({\color{gray}-1.265 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id	entity type ID: `https://blockprotocol.org/@alice/types/entity-type/book/v/1`	$$9.04 \mathrm{ms} \pm 52.4 \mathrm{μs}\left({\color{gray}-1.694 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id	entity type ID: `https://blockprotocol.org/@alice/types/entity-type/building/v/1`	$$8.76 \mathrm{ms} \pm 41.9 \mathrm{μs}\left({\color{gray}-1.626 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id	entity type ID: `https://blockprotocol.org/@alice/types/entity-type/organization/v/1`	$$9.00 \mathrm{ms} \pm 52.1 \mathrm{μs}\left({\color{gray}0.415 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id	entity type ID: `https://blockprotocol.org/@alice/types/entity-type/page/v/2`	$$9.03 \mathrm{ms} \pm 56.6 \mathrm{μs}\left({\color{gray}0.274 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id	entity type ID: `https://blockprotocol.org/@alice/types/entity-type/person/v/1`	$$8.90 \mathrm{ms} \pm 55.4 \mathrm{μs}\left({\color{gray}0.539 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id	entity type ID: `https://blockprotocol.org/@alice/types/entity-type/playlist/v/1`	$$8.96 \mathrm{ms} \pm 71.6 \mathrm{μs}\left({\color{gray}-2.311 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id	entity type ID: `https://blockprotocol.org/@alice/types/entity-type/song/v/1`	$$9.23 \mathrm{ms} \pm 87.8 \mathrm{μs}\left({\color{gray}0.594 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id	entity type ID: `https://blockprotocol.org/@alice/types/entity-type/uk-address/v/1`	$$8.88 \mathrm{ms} \pm 63.6 \mathrm{μs}\left({\color{gray}0.460 \mathrm{\%}}\right) $$	Flame Graph

representative_read_entity_type

Function	Value	Mean	Flame graphs
get_entity_type_by_id	Account ID: `bf5a9ef5-dc3b-43cf-a291-6210c0321eba`	$$7.11 \mathrm{ms} \pm 42.8 \mathrm{μs}\left({\color{gray}0.137 \mathrm{\%}}\right) $$	Flame Graph

representative_read_multiple_entities

Function	Value	Mean	Flame graphs
entity_by_property	traversal_paths=0	0	$$57.0 \mathrm{ms} \pm 490 \mathrm{μs}\left({\color{red}17.2 \mathrm{\%}}\right) $$
entity_by_property	traversal_paths=255	1,resolve_depths=inherit:1;values:255;properties:255;links:127;link_dests:126;type:true	$$98.0 \mathrm{ms} \pm 616 \mathrm{μs}\left({\color{red}6.52 \mathrm{\%}}\right) $$
entity_by_property	traversal_paths=2	1,resolve_depths=inherit:0;values:0;properties:0;links:0;link_dests:0;type:false	$$58.7 \mathrm{ms} \pm 416 \mathrm{μs}\left({\color{red}9.12 \mathrm{\%}}\right) $$
entity_by_property	traversal_paths=2	1,resolve_depths=inherit:0;values:0;properties:0;links:1;link_dests:0;type:true	$$66.0 \mathrm{ms} \pm 312 \mathrm{μs}\left({\color{red}9.41 \mathrm{\%}}\right) $$
entity_by_property	traversal_paths=2	1,resolve_depths=inherit:0;values:0;properties:2;links:1;link_dests:0;type:true	$$74.2 \mathrm{ms} \pm 410 \mathrm{μs}\left({\color{red}6.66 \mathrm{\%}}\right) $$
entity_by_property	traversal_paths=2	1,resolve_depths=inherit:0;values:2;properties:2;links:1;link_dests:0;type:true	$$78.2 \mathrm{ms} \pm 401 \mathrm{μs}\left({\color{red}5.98 \mathrm{\%}}\right) $$
link_by_source_by_property	traversal_paths=0	0	$$38.5 \mathrm{ms} \pm 222 \mathrm{μs}\left({\color{gray}2.21 \mathrm{\%}}\right) $$
link_by_source_by_property	traversal_paths=255	1,resolve_depths=inherit:1;values:255;properties:255;links:127;link_dests:126;type:true	$$63.6 \mathrm{ms} \pm 335 \mathrm{μs}\left({\color{red}5.09 \mathrm{\%}}\right) $$
link_by_source_by_property	traversal_paths=2	1,resolve_depths=inherit:0;values:0;properties:0;links:0;link_dests:0;type:false	$$43.4 \mathrm{ms} \pm 203 \mathrm{μs}\left({\color{gray}1.64 \mathrm{\%}}\right) $$
link_by_source_by_property	traversal_paths=2	1,resolve_depths=inherit:0;values:0;properties:0;links:1;link_dests:0;type:true	$$51.1 \mathrm{ms} \pm 218 \mathrm{μs}\left({\color{gray}-0.637 \mathrm{\%}}\right) $$
link_by_source_by_property	traversal_paths=2	1,resolve_depths=inherit:0;values:0;properties:2;links:1;link_dests:0;type:true	$$53.2 \mathrm{ms} \pm 284 \mathrm{μs}\left({\color{gray}0.951 \mathrm{\%}}\right) $$
link_by_source_by_property	traversal_paths=2	1,resolve_depths=inherit:0;values:2;properties:2;links:1;link_dests:0;type:true	$$53.3 \mathrm{ms} \pm 323 \mathrm{μs}\left({\color{gray}2.08 \mathrm{\%}}\right) $$

scenarios

Function	Value	Mean	Flame graphs
full_test	query-limited	$$103 \mathrm{ms} \pm 435 \mathrm{μs}\left({\color{gray}0.541 \mathrm{\%}}\right) $$	Flame Graph
full_test	query-unlimited	$$110 \mathrm{ms} \pm 339 \mathrm{μs}\left({\color{gray}0.901 \mathrm{\%}}\right) $$	Flame Graph
linked_queries	query-limited	$$20.5 \mathrm{ms} \pm 134 \mathrm{μs}\left({\color{red}35.9 \mathrm{\%}}\right) $$	Flame Graph
linked_queries	query-unlimited	$$425 \mathrm{ms} \pm 565 \mathrm{μs}\left({\color{lightgreen}-5.007 \mathrm{\%}}\right) $$	Flame Graph

Copilot AI review requested due to automatic review settings June 30, 2026 09:46

vercel Bot temporarily deployed to Preview – petrinaut June 30, 2026 09:46 Inactive

graphite-app Bot assigned indietyp Jun 30, 2026

github-actions Bot added area/libs Relates to first-party libraries/crates/packages (area) type/eng > backend Owned by the @backend team area/tests New or updated tests labels Jun 30, 2026

Copilot started reviewing on behalf of indietyp June 30, 2026 09:46 View session

cursor Bot reviewed Jun 30, 2026

View reviewed changes

Comment thread libs/@local/graph/postgres-store/src/store/postgres/knowledge/entity/mod.rs

Comment thread libs/@local/graph/store/src/embedding/clustering.rs Outdated

vercel Bot had a problem deploying to Preview – hash June 30, 2026 09:51 Failure

Copilot AI reviewed Jun 30, 2026

View reviewed changes

Comment thread libs/@local/graph/store/src/embedding/clustering.rs

Comment thread libs/@local/graph/postgres-store/src/store/postgres/knowledge/entity/mod.rs Outdated

Comment thread libs/@local/graph/api/src/rest/entity/mod.rs

Comment thread libs/@local/graph/store/src/entity/store.rs Outdated

fix: suggestions from code review

e0a451a

indietyp force-pushed the bm/be-629-implement-kmeans-clustering-in-the-hash-graph branch from bf17f16 to a9bcac4 Compare July 1, 2026 08:08

github-actions Bot added the area/deps Relates to third-party dependencies (area) label Jul 1, 2026

cursor Bot reviewed Jul 1, 2026

View reviewed changes

Comment thread libs/@local/graph/postgres-store/src/store/postgres/knowledge/entity/mod.rs

vercel Bot deployed to Preview – hashdotdesign-tokens July 1, 2026 08:10 View deployment

vercel Bot deployed to Preview – petrinaut July 1, 2026 08:12 View deployment

fix: spawn blocking for clustering

bc0b832

Copilot AI review requested due to automatic review settings July 1, 2026 08:13

vercel Bot temporarily deployed to Preview – petrinaut July 1, 2026 08:13 Inactive

Copilot started reviewing on behalf of indietyp July 1, 2026 08:13 View session

fix: regenerate

7ae58ad

vercel Bot temporarily deployed to Preview – petrinaut July 1, 2026 08:17 Inactive

Copilot AI reviewed Jul 1, 2026

View reviewed changes

Comment thread libs/@local/graph/postgres-store/src/store/postgres/knowledge/entity/mod.rs

Comment thread libs/@local/graph/store/src/entity/store.rs

Comment thread libs/@local/graph/store/src/error.rs

Copilot AI review requested due to automatic review settings July 1, 2026 08:18

indietyp added 2 commits July 6, 2026 16:40

fix: wording

3677998

chore: regen openapi

8795746

Copilot AI review requested due to automatic review settings July 6, 2026 14:42

vercel Bot temporarily deployed to Preview – petrinaut July 6, 2026 14:42 Inactive

Copilot AI reviewed Jul 6, 2026

View reviewed changes

Comment thread libs/@local/graph/store/src/error.rs Outdated

Comment thread libs/@local/graph/store/src/entity/store.rs

Comment thread apps/hash-graph/src/subcommand/mod.rs Outdated

Comment thread libs/@local/graph/embeddings/src/clustering.rs

indietyp added 4 commits July 6, 2026 18:03

chore: docs

e090faf

feat: change the accumulate clusters method

2ab2079

chore: docs

6bedce9

feat: move into embeddings crate

5e46122

cursor Bot reviewed Jul 7, 2026

View reviewed changes

Comment thread libs/@local/graph/embeddings/src/clustering.rs

fix: CI

5ff4cc0

github-advanced-security AI found potential problems Jul 7, 2026

View reviewed changes

indietyp added 2 commits July 7, 2026 10:29

chore: regenerate files

16079a8

fix: the darn yarn lockfile

77bc0a0

TimDiekmann reviewed Jul 7, 2026

View reviewed changes

Comment thread libs/@local/graph/embeddings/src/clustering.rs

TimDiekmann reviewed Jul 7, 2026

View reviewed changes

Comment thread libs/@local/graph/embeddings/src/clustering.rs

TimDiekmann reviewed Jul 7, 2026

View reviewed changes

chore: remove tautological tests

b4def7d

cursor Bot reviewed Jul 7, 2026

View reviewed changes

Comment thread libs/@local/graph/embeddings/src/clustering.rs

Copilot AI reviewed Jul 7, 2026

View reviewed changes

indietyp added 2 commits July 7, 2026 12:57

feat: only initialize once.

2a952d3

fix: docs

2f8b031

Copilot AI reviewed Jul 7, 2026

View reviewed changes

feat: move to rayon spawns

e501333

github-advanced-security AI found potential problems Jul 7, 2026

View reviewed changes

indietyp added 2 commits July 7, 2026 21:28

feat: test

f6ad162

chore: lockfile

795ea71

cursor Bot reviewed Jul 7, 2026

View reviewed changes

Copilot AI reviewed Jul 7, 2026

View reviewed changes

Uh oh!

Conversation

indietyp commented Jun 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🌟 What is the purpose of this PR?

🔍 What does this change?

Pre-Merge Checklist 🚀

🚢 Has this modified a publishable library?

📜 Does this require a change to the docs?

🕸️ Does this require a change to the Turbo Graph?

🛡 What tests cover this?

❓ How to test this?

Uh oh!

vercel Bot commented Jun 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cursor Bot commented Jun 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Summary

Uh oh!

indietyp commented Jun 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

codecov Bot commented Jul 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

codspeed-hq Bot commented Jul 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging this PR will degrade performance by 15.38%

Performance Changes

Footnotes

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

indietyp commented Jun 30, 2026 •

edited

Loading

vercel Bot commented Jun 30, 2026 •

edited

Loading

cursor Bot commented Jun 30, 2026 •

edited

Loading

indietyp commented Jun 30, 2026 •

edited

Loading

codecov Bot commented Jul 1, 2026 •

edited

Loading

codspeed-hq Bot commented Jul 1, 2026 •

edited

Loading