Skip to content

Harden Compass performance and improve semantic graph dominance - #86

Merged
forhappy merged 53 commits into
mainfrom
codex/compass-performance-hardening
Jul 30, 2026
Merged

Harden Compass performance and improve semantic graph dominance#86
forhappy merged 53 commits into
mainfrom
codex/compass-performance-hardening

Conversation

@forhappy

@forhappy forhappy commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Summary

  • add a correctness-gated performance harness for cold, warm, incremental, natural-language query, and CompassQL workloads
  • verify Compass and Graphify on pinned Django (Python/JavaScript) and Entire (Go/shell) repositories
  • preserve path-sensitive cache identity, sealed unchanged-build reuse, empty-file inventory, and deterministic byte-stable publication
  • resolve Python imported types through re-exports and retain safely qualified unresolved external member calls
  • extract bounded JavaScript prototype / .fn methods and correctly anchor grouped/generated Go types
  • classify cross-schema facts as exact, semantically dominated, ambiguous, or missing without label-only matching
  • reduce publication work through owned fact transfer, canonical ordering, bounded anchor parsing, deferred failure diagnostics, and indexed Python call sites
  • improve identifier-aware query ranking, including CamelCase symbols such as URLResolver

Why

Compass needed repeatable, fail-closed evidence for both performance and graph quality. Literal Graphify comparison mixed real omissions with cases where Compass owns a stronger canonical definition. The implementation therefore improves native extraction/resolution first, then uses Graphify only as a development oracle to prove exact coverage or bounded semantic dominance.

Root causes fixed in this phase include lexical/canonical cache-root disagreement, empty generic files being cleared after extraction, Python re-export provenance stopping short of anchored definitions, qualified external calls losing their endpoints, JavaScript prototype assignments not becoming owned methods, and Go receiver placeholders winning over later real or grouped type declarations.

Real-repository quality results

Both final graphs have zero validation errors.

Repository Graphify fact Exact Dominated Ambiguous Missing Exact + dominated
Django Nodes 50,466 6 25 348 99.27%
Django Edges 149,993 336 26 8,355 94.72%
Entire Nodes 19,866 663 6 50 99.73%
Entire Edges 53,593 6,104 18 1,347 97.76%

This improves substantially over literal missing counts of Django 1,175 nodes / 10,600 edges and Entire 720 nodes / 7,470 edges. Strict Graphify-superset quality is intentionally not claimed: the remaining cases fail closed and are concentrated in config/vendored-resource relations, shell entry points, Go embedding occurrences, and ambiguous inherited/generated identities.

The final generated-Go correction publishes ErrorModelStatusCode at its declaration and optionFunc as an anchored type_alias; on Entire it reduces ambiguous nodes from 20 to 6 and missing nodes from 52 to 50.

Performance

Pinned corpora: Django 50d706d0, Entire 279b9885. Fresh comparison: Graphify 0.9.31 at 4fe11092; historical baseline: Graphify 0.9.30 at ecfcd160.

Tool Repository Workload p50 p95 Peak RSS
Compass Django Cold, final 3 12.355s 16.587s 4.45 GiB
Compass Django Warm, final 5 2.182s 2.429s 674.58 MiB
Graphify 0.9.31 Django Cold, 3 49.985s 50.916s 1.26 GiB
Compass Entire Cold, 3 4.055s 4.127s 1.42 GiB
Compass Entire Warm, 3 0.586s 0.599s 247.25 MiB
Graphify 0.9.31 Entire Cold, 3 17.650s 18.214s 226.09 MiB

Django remains 5.05x faster than the pinned Graphify 0.9.30 median. Graphify 0.9.31 itself became materially faster, so current-head cold ratios are 4.05x on Django and 4.35x on Entire. Django warm is 11.96% above the prior 1.949s baseline, narrowly outside the 10% phase gate. Peak RSS also remains higher than Graphify. These remain explicit follow-up performance gaps.

Query verification

Single fresh-process semantic checks (not latency medians): Compass passed all four repository oracles. Graphify passed three; its Django URL-resolution result omitted required URLResolver evidence.

Query Compass Graphify 0.9.31
Django URL resolution 2.783s, pass 2.414s, fail oracle
Django model save 3.303s, pass 4.232s, pass
Entire checkpoint creation 0.655s, pass 1.199s, pass
Entire repository state 1.614s, pass 1.923s, pass

Validation

  • cargo fmt --all -- --check
  • cargo test -p compass-files --test contracts (23 passed)
  • complete compass-languages, compass-resolve, and compass-graph suites
  • graph-v1 determinism (8 passed) and publication resilience (10 passed)
  • python3 -m unittest discover -s benchmarks/performance/tests -v (60 passed)
  • release build and fresh Django/Entire graph builds with zero validation errors
  • git diff --check
  • parent graphify update . refresh completed

Detailed implementation context, exact constraints, methodology, and remaining gaps are in docs/superpowers/plans/2026-07-30-semantic-graphify-dominance.md and docs/superpowers/reviews/2026-07-30-compass-performance-baseline.md.

forhappy added 26 commits July 30, 2026 07:15
@forhappy forhappy changed the title Harden Compass build and query performance qualification Harden Compass performance and qualify graph quality against Graphify Jul 30, 2026
@forhappy forhappy changed the title Harden Compass performance and qualify graph quality against Graphify Harden Compass performance and improve semantic graph dominance Jul 30, 2026
@forhappy
forhappy marked this pull request as ready for review July 30, 2026 21:33
@forhappy
forhappy merged commit 82e73ce into main Jul 30, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant