Skip to content

feat: add Clojure structural extraction#2203

Open
bobspryn wants to merge 2 commits into
Graphify-Labs:v8from
bobspryn:feat/clojure-extractor
Open

feat: add Clojure structural extraction#2203
bobspryn wants to merge 2 commits into
Graphify-Labs:v8from
bobspryn:feat/clojure-extractor

Conversation

@bobspryn

Copy link
Copy Markdown

Summary

  • add structural extraction for Clojure, ClojureScript, shared .cljc sources,
    and Babashka scripts through a new optional clojure extra
  • extract namespaces, common definitions, protocols, record/type
    implementations, multimethods, imports, and resolved same-file calls
  • preserve Clojure's case-sensitive symbols and reader-conditional feature
    context while excluding quoted, syntax-quoted, commented, and discarded forms
  • wire the extractor through detection, dispatch, dependency warnings,
    language-family analysis, hooks, docs, and generated skill artifacts

Prior art

This revisits #816 against the current modular v8 extractor architecture.
Credit to @alendit for the original Clojure extractor and test direction. That
PR is currently conflicted with v8; this implementation keeps its intent while
updating the integration points and expanding behavioral coverage.

Notes

  • .edn remains data rather than executable Clojure source, so it is not sent
    through the code extractor.
  • tree-sitter-clojure-orchard currently ships as a source distribution.
    Installing graphifyy[clojure] therefore requires a C compiler.

Validation

  • uv sync --all-extras --frozen
  • uv run --frozen pytest tests/ -q --tb=short — 3652 passed, 3 skipped
  • uv run pre-commit run --all-files
  • all skill-generation checks, including monolith and always-on round trips
  • targeted Pyright check for the new extractor and generator changes
  • clean-environment package install plus graphify extract --code-only smoke
    test over the Clojure fixture — 24 nodes, 38 edges

bobspryn added 2 commits July 25, 2026 19:08
normalize_id() collapses every non-word run to "_" and strips it, so
`live?`, `save!` and `x*` each normalized onto the bare name. _clojure_id()
guarded only case (Foo vs foo), leaving the sigil pairs to fuse — and since
a predicate beside its plain counterpart is ordinary Clojure, the extractor
hit its own ID-collision assertion and dropped the WHOLE file from the
graph. A namespace defining `live?` next to `live` contributed nothing.

Generalize the guard from "case is significant" to "anything normalization
destroys is significant". Only `-`, `.` and `/` are treated as
information-preserving: they always mean `_`, and `_` is not idiomatic
inside a Clojure symbol, which is what makes that mapping safe. Any other
divergence between the exact name and its normalized form pins the name
with a short digest.

Kebab-case functions and dotted namespaces therefore keep their readable
IDs (`fetch-record` -> `..._fetch_record`, `nucleus.store.send-records` ->
`nucleus_store_send_records`); only genuinely ambiguous names pay for the
disambiguation. On a 215-file Clojure corpus that is 16% of nodes.

Call edges are unaffected: they resolve through definition_ids_by_name,
which is keyed on the exact source name, so a call to `live?` still reaches
`live?` and not `live`.
@bobspryn

Copy link
Copy Markdown
Author

Pushed a follow-up (8f69b18) fixing an ID-collision bug found running this extractor over a 215-file Clojure codebase.

normalize_id() collapses every non-word run to _ and strips it, so live?, save! and x* each normalized onto the bare name. _clojure_id() guarded only case (Foo vs foo), leaving the sigil pairs to fuse — and because a predicate sitting beside its plain counterpart is ordinary Clojure, this tripped the extractor's own collision assertion and dropped the whole file from the graph. One namespace defining live? next to live contributed nothing at all.

The guard is now "anything normalization destroys is significant" rather than "case is significant". Only -, . and / are treated as information-preserving — they always mean _, and _ isn't idiomatic inside a Clojure symbol, which is what makes that mapping safe. Anything else pins the exact name with a short digest.

Kebab-case functions and dotted namespaces keep their readable IDs (fetch-record..._fetch_record, nucleus.store.send-recordsnucleus_store_send_records); only genuinely ambiguous names pay for it — 16% of nodes on that corpus. Call edges are unaffected, since they resolve through definition_ids_by_name, keyed on the exact source name.

Two tests added: one for sigil distinction plus call routing (a call to live? must reach live?, not live), one asserting kebab/dotted names don't get a digest. Both confirmed to fail on the pre-fix extractor. Corpus coverage went 213/214 → 215/215 files, zero skips.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant