Fix intermittent key loss in SeSaveRecoverMultipleKeysTest("25k","18k")

[TedHartMS]

This pull request addresses a regression in snapshot recovery under memory budget constraints, ensuring that all pages are properly flushed and no data is lost during recovery. It also improves resource management and clarifies code paths for different store configurations. The most important changes are:

Bug Fixes and Recovery Logic Improvements:

• Modified RecoverSnapshotPages in Recovery.cs to always flush every page when a LogSizeTracker is attached, regardless of whether the store is an object allocator or not. This prevents data loss when recovering into a smaller memory budget, fixing a scenario where pages could be evicted before being flushed, resulting in lost records.
• Simplified the call to RecoverSnapshotPages by removing unused parameters related to page ranges, reflecting the new logic that flushes all pages under a memory budget. [1] [2]

Resource Management:

• Updated disposal logic to safely handle cases where objectLogRecoveryDevice may be null, preventing potential null reference exceptions.
• Improved initialization of objectLogRecoveryDevice to only occur when the store has an object log, and made its initialization call null-safe.

Testing and Documentation:

• Added a comprehensive regression test SpanByteRecoverySnapshotEvictionTests to verify that all records are correctly recovered when restoring from a snapshot under a tighter memory budget, and that no records are lost due to premature eviction.
• Added clarifying comments in ContinuePending.cs to document status handling during read operations.

[Copilot]

Pull request overview

This PR fixes an intermittent snapshot-recovery data-loss regression under tight memory budgets by ensuring snapshot pages are made durable (flushed) before recovery-time eviction can free them. It also adds a regression test covering the failure mode and tightens a couple of recovery/resource-management code paths.

Changes:

• Update Tsavorite snapshot recovery to flush every snapshot page when a LogSizeTracker is attached (so recovery-time eviction cannot drop unflushed pages).
• Make snapshot object-log recovery device initialization/disposal null-safe when the store has no object log.
• Add a regression test that recovers a SpanByte snapshot into a smaller budget (with optional concurrent resizer) and verifies all records read back correctly; add a clarifying comment in ContinuePendingRead.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File	Description
libs/storage/Tsavorite/cs/src/core/Index/Recovery/Recovery.cs	Flush all snapshot pages under LogSizeTracker; null-safe object-log recovery device setup/teardown.
libs/storage/Tsavorite/cs/test/test.recovery/SpanByteRecoverySnapshotEvictionTests.cs	New regression test for snapshot recovery under budget-driven eviction (SpanByte/inline store).
libs/storage/Tsavorite/cs/src/core/Index/Tsavorite/Implementation/ContinuePending.cs	Comment clarifying why status is already SUCCESS at the copy-to-tail/readcache decision point.

[hexonal]

Possibly-related data point: I've seen the same works-before-save, one-record-missing-after-tryRecover symptom show up in an unrelated code path — RespVectorSetTests.RecreateIndexesOnRestoreAsync (test/standalone/Garnet.test.vectorset/RespVectorSetTests.cs:2348) — twice today, on two different unrelated PR branches, both freshly rebased onto current main:

• https://github.com/microsoft/garnet/actions/runs/29979858823/job/89123471985 (Garnet Standalone (ubuntu-latest, net8.0, Release, Garnet.test.vectorset))
• https://github.com/microsoft/garnet/actions/runs/29980825950/job/89122569138 (same job name)

Both fail identically: after VADDing 2 elements to a vector set, a VSIM ... ELE ... query correctly returns 2 matches, then a ForegroundSave + WaitForCommitAsync + server restart with tryRecover: true, and the same query only returns 1 match (Expected: 2, But was: 1). Deterministic across both runs, not a random/timing-shaped flake.

I haven't been able to bisect or reproduce this locally (no native diskann build available on macOS/arm64 in my sandbox, so every VADD fails locally regardless of branch), so take this as a raw observation, not a confirmed diagnosis. But the shape matches this PR's fix description closely: VectorSet element records live in the main store as fixed-size stub records, and RecoverSnapshotPages's old behavior (skip-flush for non-object-allocator stores) would apply to that store too. If a stub record's page got evicted before being flushed during recovery, the RecreateIndex-on-first-access path would rebuild the native HNSW index from whatever main-store records actually survived — silently dropping the evicted element, which matches "1 of 2 elements found" exactly.

Not asking you to scope this PR to cover it — just flagging in case it's useful corroborating evidence (or in case you want to add a VectorSet-specific case to SpanByteRecoverySnapshotEvictionTests once this lands). Happy to help verify once #1957 merges if that'd be useful.