Per-platform custom library structure with non-destructive identity

[gantoine]

Closes #2050. A per-platform custom library structure system, modeled on Retrom's, with content-hash (non-destructive) identity so reorganizing a library doesn't orphan saves/favorites/history/collections.

Supersedes the earlier scan.subfolders flag (also on this branch) — that's been replaced by the template system per maintainer direction.

What it does

By default a platform's ROM folder is scanned one level deep (each top-level file = a game, each top-level folder = one multi-file game). A template lets you describe a deeper/organized layout instead. Opt-in per platform by fs_slug; platforms left out keep the default behavior, so existing setups are unaffected. No DB schema change.

filesystem:
  structure:
    ps3: "{category}/{gameDir}"    # roms/ps3/Disc/Game/, roms/ps3/PSN/Game/
    snes: "{region}/{gameFile}"    # roms/snes/USA/foo.sfc

Template syntax (Retrom-style)

A /-separated sequence of path sections, relative to the platform's ROM folder (RomM already resolves the library root, roms_folder, and platform directory — so {library}/{platform} aren't used here and are rejected with a helpful message):

• a braced section is a macro; a bare section is a literal folder matched exactly;
• the last section must be a terminal: {gameFile} (each file is a game) or {gameDir} (each folder is a single multi-file game);
• any other braced section ({region}, {category}, …) is a wildcard directory level — matches any folder, purely organizational.

Declaring {gameFile} vs {gameDir} removes the disc-descriptor guesswork (the cue/bin ambiguity from #3565): the user states file-vs-folder explicitly. Hidden (dot-prefixed) folders are never descended into.

Mixed layouts (list of templates)

A single fixed-depth template can't describe a platform that has loose games at the root and grouping subfolders below it — a very common setup. So a platform's value may be a list of templates; discovery is their union, deduplicated by full path:

filesystem:
  structure:
    nes:
      - "{gameFile}"             # loose top-level games
      - "{category}/{gameFile}"  # games inside grouping subfolders (Hacks/, …)

This composes freely (mix file/dir terminals, multiple depths) without reintroducing arbitrary recursion.

Non-destructive identity

Identity is reconciled by content hash during the scan: moving or renaming a game within the structure is matched against a now-missing ROM and relocated in place, so its saves, play history, favorites and collection membership follow it. Falls back to path identity when hashing is unavailable (skip_hash_calculation / non-hashable platform). A cheap size pre-filter avoids hashing unrelated files. Only runs for platforms with a custom structure.

Frontend

The Files tab shows a ROM's on-disk Location (click-to-copy) — useful for nested layouts. Built as a shared LocationChip mirroring HashChip's RTag-based pattern. New i18n keys in all locales; parity check passes.

Implementation

• config: parse_library_structure / parse_platform_structures / LibraryStructure, validated at load; platform_structure() returns the tuple of structures to union. Accepts str | list[str].
• fs handler: _discover_structured_roms walks literal/wildcard levels to the terminal; _collect_fs_roms unions across a platform's structures (dedup by full path); _discover_default_roms preserves default behavior. Each ROM records its real fs_path.
• db: get_roms_by_fs_name / mark_missing_roms key on full path so identically-named files in different folders stay distinct; get_roms_for_relocation for hash matching.

Testing

• Backend: template parser (valid/invalid, string + list forms), per-platform config loading, structured discovery (wildcard/literal levels, file/dir terminals, depth > 1, hidden-folder skip, cross-folder collisions, mixed loose+grouped union), full-path keying, and hash-based relocation.
• Suite green except pre-existing, environment-only failures that also fail on master (a CHD test relying on chmod 0o000 denying reads — impossible as root; one auth test needing live Redis; two fulltext-search ranking tests).
• Frontend: vue-tsc, eslint, full vitest, and i18n parity all pass.

AI disclosure: authored with Claude Code, driven and reviewed by the maintainer.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Sk2dzM3K9qxWBdPAeGb7us

[androosio]

Hey @gantoine, the hash identity is clearly a better approach than mine 👍 However, the templating is a bit too restrictive in its current form and doesn't take into account libraries that contain roms in the main platform folder and 'grouping' subfolders below that. I'm sure I'm not the only one with a setup like this, e.g.

/nes
  game 01
  game 02
  Hacks subfolder
    game 03
    game 04

templates:

• {gameFile} > the loose top-level games, but nothing inside the grouping folders.
• {category}/{gameFile} > the grouping folders' contents, but every loose top-level game is dropped.

With the template approach, I'd have to add at least one subfolder to each platform containing just the top-level roms

My scan.subfolders: true in #3565 was the middle-ground option to handle libraries with my structure. Hoping your implementation can keep that flexibility.

[gantoine]

@androosio check "Mixed layouts" section and lmk if that works for ya?

[github-actions]

Test Results (postgresql)

    1 files  ± 0      1 suites  ±0   2m 29s ⏱️ -2s
1 812 tests +25  1 812 ✅ +25  0 💤 ±0  0 ❌ ±0 
1 814 runs  +25  1 814 ✅ +25  0 💤 ±0  0 ❌ ±0 

Results for commit 4df54ce. ± Comparison against base commit 462822c.

♻️ This comment has been updated with latest results.

[github-actions]

Test Results (mariadb)

    1 files  ± 0      1 suites  ±0   2m 12s ⏱️ -15s
1 812 tests +25  1 812 ✅ +25  0 💤 ±0  0 ❌ ±0 
1 814 runs  +25  1 814 ✅ +25  0 💤 ±0  0 ❌ ±0 

Results for commit 4df54ce. ± Comparison against base commit 462822c.

♻️ This comment has been updated with latest results.

[github-actions]

☂️ Python Coverage

current status: ✅

Overall Coverage

Lines	Covered	Coverage	Threshold	Status
19018	13814	73%	0%	🟢

New Files

No new covered files...

Modified Files

File	Coverage	Status
backend/config/config_manager.py	67%	🟢
backend/endpoints/sockets/scan.py	30%	🟢
backend/handler/database/roms_handler.py	68%	🟢
backend/handler/filesystem/roms_handler.py	89%	🟢
TOTAL	63%	🟢

updated for commit: 4df54ce by action🐍

[androosio]

@androosio check "Mixed layouts" section and lmk if that works for ya?

Perfect! thanks very much.

[thekiefs]

@gantoine thanks much for pushing this out way faster than I could get to it! I started something here: https://github.com/thekiefs/romm/tree/feature/dynamic-libraries ... but I think your hash approach is much more elegant :)

Slight note, I was designing for two things I don't think this PR covers.

1. this looks like it scopes templates to within a platform's ROM folder. RomM still resolves the library root → roms_folder → platform directory itself, using the existing conventions. I see {library} and {platform} aren't valid template variables.
2. this also doesn't cover multiple libraries on different root disks - but I realize this is probably a rarely needed feature

For 1, the path it can handle looks like:
{LIBRARY_BASE_PATH}/roms/Atari - 2600/{whatever the template describes}

But it can't handle:
{LIBRARY_BASE_PATH}/data/games/Atari - 2600

/roms/ is what every rommapper uses today and I understand you'd like to keep existing path resolution, but this might also cause new users some heartburn - especially those with large libraries and existing setups/applications built around their existing directory.

Instead, could you offer a per-platform override that looks something like this:

filesystem:
  structure:
     atari-2600:
      path: "/data/games/Atari - 2600"

[gantoine]

@thekiefs totally reasonable, will implement some version of that! (when i circle back to this after 5.0)