Add LoRA analysis: SVD spectrum, effective rank, Frobenius norms#1
Closed
harelc wants to merge 4 commits into
Closed
Add LoRA analysis: SVD spectrum, effective rank, Frobenius norms#1harelc wants to merge 4 commits into
harelc wants to merge 4 commits into
Conversation
- LoRA pair detection with support for common naming conventions - Tensor data loading and SVD via QR-based optimization on small rank×rank matrix - LoraScreen: full-screen DataTable with rank, ||A||/||B|| norms, effective rank - SvdScreen: vertical bar histogram (√ scale) with rank statistics - LoraHelpScreen (?) explains stable rank, Frobenius norm, SVD spectrum - Context-aware L key: filters to visible pairs, jumps to SVD for single match - Async 4-thread pool for progressive stats computation, cached across screen opens - Load/export analysis as JSON (e key); auto-loads on startup if present - Clean thread shutdown on quit and screen dismiss - numpy is a required dependency Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
9409d05 to
94579b9
Compare
- Add CompactifyScreen to TUI: LoRA screen -> 'c' -> enter target rank to truncate all pairs via SVD and save a smaller safetensors file - Add truncate_lora_pair(), load_all_tensors(), write_safetensors() to data.py - Fix critical SVD bug in compute_lora_svd(): QR of b.T had wrong dimensions, silently fell back to full (4096x4096) matrix SVD instead of (64x64). Now uses qr(b) correctly — ~1000x faster effective rank computation. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Replace read-only MetadataScreen with interactive editor: field-by-field view, edit (e), add (a), delete (d), save (Ctrl+S) - Add ScaleScreen for scaling LoRA pairs by alpha coefficient (accessible via 'a' key from LoRA analysis view) - Add scale_lora_a() and rewrite_with_metadata() to data.py - Load sigma0 from cached JSON on startup Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add interactive metadata editor and LoRA scaling
Owner
|
Hi @harelc — thanks a lot for putting this together, the QR-SVD trick and the per-pair stats screen were both great ideas. I ended up reimplementing the core pieces against the new multi-command structure rather than merging the branch directly, but most of the work is now in v0.2.0: What landed (inspired by this PR):
Not (yet) backported, in case you want to follow up:
Closing this PR since the core features are in v0.2.0; happy to take a follow-up PR for any of the gaps above. Thanks again! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.