Complete Dock 12.0.0.2 -> 12.1.0.2 upgrade: migrate legacy $id/$ref layouts that break the new reference handling

[JoshuaRowePhantom]

Summary

The Dock.* pins are held at 12.0.0.2 because upgrading to 12.1.0.2 breaks deserialization of already-persisted ("legacy") dock layouts. Dock 12.1.0.2 (upstream PR danipen/Dock #1107 "Fix Owner reference-cycle handling across all serializers") widened the System.Text.Json $id/$ref reference-tracking scope so it no longer resets per-list. Legacy layouts written by 12.0.0.2 contain massively duplicated $id values (a per-list-isolated-scope artifact — the very bloat from #1335/#1334), which the 12.1.0.2 deserializer rejects. Impact: we cannot take the Dock upgrade (and its native #1107 reference-cycle fix that supersedes our #1335 workaround) until legacy persisted layouts are migrated. This is also a real user-facing MIGRATION risk: any user's already-persisted 12.0.0.2 layout would fail to load after the upgrade, not just the test fixture.

Root Cause

• Failing test: features\Phantom.Workspaces.Tests\WorkspaceDockPersistenceTests.cs:132-150 (WorkspaceDockPersistence_LoadRealLayout1334_HealsTo11Docs). Loads features\Phantom.Workspaces.Tests\Fixtures\dock-layout-1334.json (~118 KB real bloated layout from Restored multi-region ProportionalDock inner layout renders both regions headerless — real tabs present but header items missing (duplicate-instance last-writer-wins wiring from persistence leak + non-atomic restore of unresolvable descriptors) #1334) via DockLayoutCanonicalizer.Deserialize(json) (:140) → Canonicalize(..., liveTabIds:null) (:143) → MigrateBaseDocumentDocksToWorkspaceContentDock (:144); asserts AllDocs.Count==11, one MvvmControls.RootDock, zero raw MvvmControls.DocumentDock, no orphan windows (:146-149).
• Serialization stack: features\Phantom.Workspaces\ViewModels\DockLayoutCanonicalizer.cs — CreatePreserveOptions() (:42-52) sets ReferenceHandler.Preserve + our WorkspaceDockTypeInfoResolver; IsPreserveFormat(json) (:60-61) = json.Contains("\"$values\""); Deserialize(json) (:68-83) uses the preserve path for new files but for LEGACY files (no $values) falls to new DockSerializer(typeof(ObservableCollection<>), new WorkspaceDockTypeInfoResolver()).Deserialize<IRootDock>(json) (:81-82) — THIS is the path the 1334 fixture takes. WorkspaceDockTypeInfoResolver (...\WorkspaceDockTypeInfoResolver.cs) applies $type discriminators and orders collections ($id) before scalar refs ($ref) via ApplySerializationOrder (:144-157).
• Fixture evidence (the smoking gun): dock-layout-1334.json has NO $values envelope (→ legacy branch) and contains 233 $id tokens but only 13 distinct values — $id="1" appears 105 times, $id="2" 101 times, "3" 13, "4" 4, "5" 2 — with only 3 $ref tokens total. Pre-Dock-layout persistence leak: ActiveDockable/DefaultDockable/FocusedDockable and orphan floating Windows are inline-cloned every save (DockSerializer bypasses ReferenceHandler.Preserve), compounding to 100+ WorkspaceDocument instances for 11 tabs #1335, DockSerializer.JsonConverterList serialized each IList<T> in an ISOLATED JsonSerializer.Serialize scope, so ReferenceHandler.Preserve's $id counter reset at every list boundary (documented in DockLayoutCanonicalizer.cs:19-26 and WorkspaceDockTypeInfoResolver.cs:23-26). Under 12.0.0.2 the deserializer mirrored that per-list scope so the reused ids were legal. Under 12.1.0.2, PR Crash: ObservableCollection reentrancy opening a tab — remove dock->Tabs order back-sync (Tabs order must be independent of dock) #1107 no longer resets the reference scope per list, so the first duplicate (105 objects sharing $id="1") throws '$id' metadata property '1' conflicts with an existing identifier. This is root cause (a), confirmed by direct fixture inspection; the dual-discriminator and "new objects gain $id" hypotheses are not needed to explain it.

Affected Files

File	Role
features\Directory.Packages.props (:17-21)	Pins all five Dock.* at 12.0.0.2 (only CPM file under features\; no submodule variant) — to move to 12.1.0.2
features\Phantom.Workspaces\ViewModels\DockLayoutCanonicalizer.cs (:42-52,60-61,68-83,86-113)	Preserve options, legacy-format detection, legacy DockSerializer deserialize path — migration pre-pass goes here
features\Phantom.Workspaces\ViewModels\WorkspaceDockTypeInfoResolver.cs (:128-157,144-157,195-208)	Custom resolver ($type + $id/$ref ordering) — candidate for simplification once #1107 native handling is trusted
features\Phantom.Workspaces\ViewModels\MainWindowViewModel.cs (:2840 write, :3583/:3600 read)	Persistence load/save call sites
features\Phantom.Workspaces.Tests\WorkspaceDockPersistenceTests.cs (:132-150)	Failing test + Expected-Tests home
features\Phantom.Workspaces.Tests\Fixtures\dock-layout-1334.json	Legacy bloated fixture with duplicate $ids

Design / Fix (ordered options)

1. PRIMARY — legacy-layout migration pre-pass in DockLayoutCanonicalizer.Deserialize before the legacy DockSerializer.Deserialize call (:81): run a JSON-level transform that either (a) STRIPS all $id/$ref metadata from legacy-format JSON (only 3 $refs total — negligible sharing lost; Dock rebuilds from $type + inline objects), or (b) RENUMBERS $ids to be globally unique and rewrites the 3 $ref targets. Pure JSON transform, no Dock-internals dependency. Crucially this also unblocks any USER's already-persisted 12.0.0.2 layout on upgrade, not just the fixture.
2. Regenerate the fixture under the current preserve write path (SerializeCanonical) as a secondary safety net so the test also covers the new persisted format; keep the legacy fixture behind a dedicated migration test.
3. Bump the pins in Directory.Packages.props:17-21 to 12.1.0.2 for all five Dock.* together (never a partial subset — Bump Dock.Avalonia and Dock.Avalonia.Themes.Fluent #1339 only moved 2 of 5, which desyncs the set).
4. Follow-up cleanup (after 1 lands): PR Crash: ObservableCollection reentrancy opening a tab — remove dock->Tabs order back-sync (Tabs order must be independent of dock) #1107 fixes the upstream per-list-isolation/Owner-cycle problem natively, so the heal/dedupe + ApplySerializationOrder machinery in WorkspaceDockTypeInfoResolver (and possibly CreatePreserveOptions) MAY be simplified toward Dock.Serializer.SystemTextJson's own serializer. Do NOT remove the heal/dedupe wholesale — it is still needed to reduce already-bloated persisted graphs (Dock-layout persistence leak: ActiveDockable/DefaultDockable/FocusedDockable and orphan floating Windows are inline-cloned every save (DockSerializer bypasses ReferenceHandler.Preserve), compounding to 100+ WorkspaceDocument instances for 11 tabs #1335). Note this simplification alone does NOT fix Review, validate, and merge all open Dependabot dependency-update PRs #1346 because the failing input is a pre-existing legacy file — option 1 is still required.
5. When implementing, check whether Dock.Serializer.SystemTextJson 12.1.0.2 exposes a ctor/option to opt out of the new reference tracking (possible temporary escape hatch, not the long-term fix).

Verification: after migration + pin bump, WorkspaceDockPersistence_LoadRealLayout1334_HealsTo11Docs passes under 12.1.0.2; full dock persistence/tab-switch/layout suites green incl. stability run.

Expected Tests

Follow WorkspaceDockPersistenceTests convention Subject_Scenario_ExpectedOutcome (existing examples at :65,:93,:112,:135).

Test Name	Class	What It Verifies
WorkspaceDockPersistence_LoadLegacyLayoutWithDuplicateIds_MigratesAndHealsTo11Docs	WorkspaceDockPersistenceTests	The legacy dock-layout-1334.json (105× $id="1") deserializes under Dock 12.1.0.2 via the migration pre-pass and still heals to 11 docs
WorkspaceDockPersistence_Dock1210Upgrade_DeserializesLegacyFixtureWithoutIdConflict	WorkspaceDockPersistenceTests	Deserializing a legacy duplicate-$id layout does not throw the '$id' ... conflicts with an existing identifier error
DockLayoutCanonicalizer_LegacyJsonWithDuplicateIds_StripsOrRenumbersReferenceMetadata	WorkspaceDockPersistenceTests (or a canonicalizer-focused test class)	The migration pre-pass produces JSON with globally-unique (or absent) $ids and intact $ref targets

Related issues

• Follow-up to Review, validate, and merge all open Dependabot dependency-update PRs #1346 (the Dependabot batch where Dock.* was HELD; this bug completes that held item — Review, validate, and merge all open Dependabot dependency-update PRs #1346 recorded the HOLD decision).
• Relates to Dock-layout persistence leak: ActiveDockable/DefaultDockable/FocusedDockable and orphan floating Windows are inline-cloned every save (DockSerializer bypasses ReferenceHandler.Preserve), compounding to 100+ WorkspaceDocument instances for 11 tabs #1335 (dock-persistence leak; introduced DockLayoutCanonicalizer/WorkspaceDockTypeInfoResolver; upstream PR Crash: ObservableCollection reentrancy opening a tab — remove dock->Tabs order back-sync (Tabs order must be independent of dock) #1107 in 12.1.0.2 is the native fix that this upgrade adopts).
• Relates to Restored multi-region ProportionalDock inner layout renders both regions headerless — real tabs present but header items missing (duplicate-instance last-writer-wins wiring from persistence leak + non-atomic restore of unresolvable descriptors) #1334 (source of the real bloated layout fixture dock-layout-1334.json).
• Upstream reference: danipen/Dock PR Crash: ObservableCollection reentrancy opening a tab — remove dock->Tabs order back-sync (Tabs order must be independent of dock) #1107 "Fix Owner reference-cycle handling across all serializers" (shipped in Dock 12.1.0.2).

[JoshuaRowePhantom]

Implementation notes

Commit: f7e9e39

Approach

Chose option 1(a) — STRIP all $id metadata and drop {"$ref":...} reference
objects from legacy-format (no $values) JSON in a pure JSON pre-pass
(DockLayoutCanonicalizer.MigrateLegacyReferenceMetadata) that runs in
Deserialize(...) before the legacy DockSerializer call. Rationale: the real
#1334 fixture has 233 $id tokens (only 13 distinct, $id="1" x105) but only 3
$ref tokens, all pointing at AD/DD/FD/Window back-references — exactly the
references the existing Canonicalize heal pass rebuilds/prunes. So stripping
loses no meaningful sharing, whereas global renumbering (option b) would require
replicating Dock's per-list $id scope resolution to disambiguate which of the
101 $id="2" objects each $ref="2" targets — strictly more complex and
error-prone. Keyed off legacy-format detection (not the fixture) so any user's
persisted 12.0.0.2 layout migrates on upgrade.

Escape hatch (option 5)

Dock.Serializer.SystemTextJson 12.1.0.2 DockSerializer exposes only ctors
taking Type and/or IJsonTypeInfoResolver — no JsonSerializerOptions /
ReferenceHandler opt-out. No supported escape hatch exists; the migration
pre-pass is required regardless (the failing input is a pre-existing legacy file).

Scope

• Bumped all five Dock.* pins in Directory.Packages.props 12.0.0.2 -> 12.1.0.2.
• Also bumped the standalone DockInspect tool (not in the solution) for set consistency.
• Did NOT touch the heal/dedupe + ApplySerializationOrder machinery (option 4 is a follow-up).
• Kept the legacy fixture dock-layout-1334.json as the migration regression input.

Files changed

• Directory.Packages.props
• DockInspect/DockInspect.csproj
• Phantom.Workspaces/ViewModels/DockLayoutCanonicalizer.cs
• Phantom.Workspaces.Tests/WorkspaceDockPersistenceTests.cs

Tests

• WorkspaceDockPersistence_LoadRealLayout1334_HealsTo11Docs now PASSES under 12.1.0.2.
• Added WorkspaceDockPersistence_LoadLegacyLayout1334UnderDock1210_Succeeds,
  WorkspaceDockPersistence_Dock1210UpgradeWithDuplicateIds_DeserializesWithoutConflict,
  DockLayoutCanonicalizer_LegacyJsonWithDuplicateIds_MigratesToConflictFreeJson.
• Targeted: WorkspaceDockPersistenceTests 7/7; broader dock/layout suite 138/138.
• Full fast suite: 5777 passed, 0 failed.

[JoshuaRowePhantom]

Verification result

Status: ✅ Verified

Commits checked:

f7e9e391 Fix #1348: migrate legacy dock layouts and complete Dock 12.0.0.2 -> 12.1.0.2 upgrade

What was verified against the committed diff (f7e9e39)

1. Migration pre-pass — MigrateLegacyReferenceMetadata + StripReferenceMetadata added to DockLayoutCanonicalizer.cs. Invoked ONLY on the legacy branch (Deserialize line 83) around MigrateLegacyReferenceMetadata(json) before the DockSerializer.Deserialize<IRootDock> call. Keyed off legacy-format detection (!IsPreserveFormat = absence of \), NOT the 1334 fixture, so any user's persisted 12.0.0.2 layout migrates on upgrade. Approach = option 1a (STRIP all \ / drop \ objects). The preserve/\ path (lines 71-80) is unchanged and the strip does not run on new-format layouts. ✅
2. Pin bump completeness — all FIVE Dock.* pins in Directory.Packages.props bumped 12.0.0.2 → 12.1.0.2, plus the standalone DockInspect/DockInspect.csproj (3 refs). Repo grep confirms NO Dock.* package pin left at 12.0.0.2 (remaining 12.0.0.2 matches are only historical design-doc/source-tag references and code comments, not pins). No Bump Dock.Avalonia and Dock.Avalonia.Themes.Fluent #1339-style partial desync. ✅
3. Tests — the 3 Expected Tests were added and genuinely exercise red→green behavior:
   • WorkspaceDockPersistence_LoadLegacyLayout1334UnderDock1210_Succeeds (legacy fixture, no \, loads).
   • WorkspaceDockPersistence_Dock1210UpgradeWithDuplicateIds_DeserializesWithoutConflict (duplicate \="1" JSON that the 12.1.0.2 deserializer would reject without the pre-pass → asserts no exception).
   • DockLayoutCanonicalizer_LegacyJsonWithDuplicateIds_MigratesToConflictFreeJson (asserts migrated JSON has no \/\ yet remains valid JSON carrying the \ discriminators).
     The existing WorkspaceDockPersistence_LoadRealLayout1334_HealsTo11Docs still runs under the bumped Dock and asserts 11 docs / 1 RootDock / 0 raw DocumentDock / no orphan windows. Not tautological. ✅
4. Heal/dedupe preserved — the Dock-layout persistence leak: ActiveDockable/DefaultDockable/FocusedDockable and orphan floating Windows are inline-cloned every save (DockSerializer bypasses ReferenceHandler.Preserve), compounding to 100+ WorkspaceDocument instances for 11 tabs #1335 heal/dedupe + ApplySerializationOrder machinery in WorkspaceDockTypeInfoResolver was NOT touched (option-4 cleanup is a follow-up, per the issue). ✅

CODE-VERIFICATION criteria: no disabled/quarantined tests, no unresolved TODOs, no timing-dependent tests, no fixed dispatcher pumping, no synchronous task blocking. Feature/test/branch coverage satisfied.

Outstanding gaps

None — all criteria satisfied.

Tests

Passed!  - Failed:     0, Passed:     7, Skipped:     0, Total:     7, Duration: 2 s - Phantom.Workspaces.Tests.dll (net10.0)
=== Test Run Summary ===
Executed : 7
Passed   : 7
Failed   : 0

This diff bumps Dock.* package pins and changes the serialization/persistence layer (filesystem-backed dock-layout load/save), so needs-slow-tests is applied to run the full slow suite at checkin.

[JoshuaRowePhantom]

Checked in

Commit: f7e9e39
Branch fast-forwarded: features → f7e9e39

Full (slow) suite: 5959 executed, 5959 passed, 0 failed. The only anomaly was the known pre-existing flake #791 (AgentFactoryTests.CreateAgentChat_UsesAgentDefinitionChatHistoryTool SlowDocker/MongoDB hang) which produced a hang dump and host crash with no test failures — treated as unrelated. Stability run of WorkspaceDockPersistenceTests: 10/10 passed.