Skip to content

feat(build): wire Jedi enrichment into builds; fix impact-accuracy benchmark - #823

Open
B1ZZ211 wants to merge 2 commits into
tirth8205:mainfrom
B1ZZ211:feat/jedi-enrichment-wiring
Open

feat(build): wire Jedi enrichment into builds; fix impact-accuracy benchmark#823
B1ZZ211 wants to merge 2 commits into
tirth8205:mainfrom
B1ZZ211:feat/jedi-enrichment-wiring

Conversation

@B1ZZ211

@B1ZZ211 B1ZZ211 commented Aug 4, 2026

Copy link
Copy Markdown

Summary

This PR wires the [enrichment] extra (Jedi call-resolution) into the build pipeline and fixes the co-change benchmark mode that was always returning predicted_files = 0.

Changes

Core wiring (code_review_graph/incremental.py)

  • Added _run_jedi_enrichment(store, repo_root) — best-effort wrapper matching the resolver pattern
  • Called in full_build (always) and incremental_update (when Python files changed)
  • Stats surfaced as result["jedi_enrichment"] (e.g. {"resolved": 5, "files": 1})
  • Opt-out via CRG_DISABLE_ENRICHMENT=1 (keeps watch mode cheap on large repos)

Bug fix in code_review_graph/jedi_resolver.py

  • Target paths now rebase onto the repo root as given to full_build (not force-resolved)
  • Fixes /var vs /private/var symlink mismatches where enriched edges never matched stored node qualified names

Benchmark fix (code_review_graph/eval/benchmarks/impact_accuracy.py)

  • Normalizes changed-file paths to the stored absolute form before lookups
  • Co-change mode now emits honest non-zero predictions (was 0 on every commit)
  • Graph-derived ground truth now includes real one-hop neighbors (recall drops from pinned 1.0 to measured 0.667 where neighbors exist)
  • _graph_neighbor_files now counts File-node-sourced edges (IMPORTS_FROM, module-level calls) as neighbor files

Tests (tests/test_incremental.py, tests/test_eval.py)

  • test_full_build_wires_jedi_enrichment — verifies wiring + stats surface
  • test_full_build_jedi_enrichment_skips_when_disabled — opt-out works
  • test_incremental_update_wires_jedi_enrichment — runs only on Python changes
  • Mock regression test asserts predicted_files > 0 and true_positives >= 1 for co-change

Verification

  • Full test suite: 2400 passed (1 pre-existing Windows test failure, unrelated)
  • Build output: Jedi enrichment: resolved 5 calls in 1 files
  • Co-change mode on real repos now reports honest numbers (previously always 0)
  • All changes follow existing resolver patterns and conventions

Notes

  • Requires [enrichment] extra (jedi>=0.19.2) — optional, skipped gracefully
  • The published eval numbers in README are now stale; a fresh 6-repo capture would be needed to re-quote them

MrB1ZZ added 2 commits August 4, 2026 05:24
…nchmark

- full_build and incremental_update now run Jedi call-resolution
  enrichment after the other resolvers. Best-effort: skipped without the
  [enrichment] extra or with CRG_DISABLE_ENRICHMENT=1; stats surface as
  result['jedi_enrichment'].
- jedi_resolver: rebase resolved target paths onto the repo root as given
  to full_build, so enriched CALLS edges match stored node qualified
  names (fixes /var vs /private/var mismatches).
- impact_accuracy: normalize changed-file paths to the stored absolute
  form so seed/neighbor/flow lookups hit; count File-node-sourced
  (module-level and IMPORTS_FROM) edges as neighbor files.
- tests: full_build/incremental_update wiring regression tests and the
  co-change mock guard.
…chment

- Remove 'harness needs fixing' note (co-change mode now works)
- Clarify why published eval CSVs show predicted_files=0 (non-code seeds)
- Update limitations section to reflect working co-change mode
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants