Skip to content

fix(io): make InMemoryFileStore.list respect directory boundaries#3977

Draft
tomsen-ai wants to merge 1 commit into
OpenHands:mainfrom
tomsen-ai:fix/inmemory-filestore-list-boundaries
Draft

fix(io): make InMemoryFileStore.list respect directory boundaries#3977
tomsen-ai wants to merge 1 commit into
OpenHands:mainfrom
tomsen-ai:fix/inmemory-filestore-list-boundaries

Conversation

@tomsen-ai

@tomsen-ai tomsen-ai commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

HUMAN:

The in-memory file store's list() disagreed with the local (disk) one:

it invented phantom directories from sibling names, and crashed with an
IndexError when given an exact file path. I went through the
side-by-side repro before and after the fix myself.


AGENT:

Why

InMemoryFileStore.list uses bare string prefix matching (file.startswith(path) + removeprefix), which is not aware of the / directory boundary. It diverges from LocalFileStore.list (whose implementation is explicitly commented as S3-consistent behavior) in two observable ways:

  1. Phantom directories from sibling keys. With keys sessions/abc/event.json and sessions/abcdef/other.json, list("sessions/abc") returns a phantom sessions/abc/def/ entry derived from the sibling's suffix. LocalFileStore returns only the real children.
  2. Crash on exact file paths. list("sessions/abc/event.json") raises IndexError: list index out of range (suffix=""parts=[]parts[0]). LocalFileStore returns [path].

Both classes implement the same FileStore interface, so consumers switching between the in-memory and local backends currently see different results for identical inputs.

The in-repo consumer (conversation/event_store.py) happens to be shielded by its EVENT_NAME_RE filename filter and fixed-width event IDs, so this is not currently user-visible there — but the divergence is a trap for any other FileStore consumer.

Summary

  • Short-circuit exact file hits to return [path], matching LocalFileStore.
  • Normalize the search prefix to end with / so listing a/b cannot match sibling keys like a/bc.txt.
  • Skip empty suffixes; keep root listing ("") behavior unchanged.
  • Add a consistency test suite that runs the same fixture through both backends and asserts they agree (red on main for the two facets above, green with this change).

Issue Number

N/A (searched existing issues/PRs: #3163/#3146 are about LRU eviction in this class, not list semantics).

How to Test

uv run pytest tests/sdk/io/ -q

New tests fail on main:

FAILED test_list_exact_file_returns_path_and_does_not_raise  (IndexError)
FAILED test_list_sibling_prefix_produces_no_phantom_entries
FAILED test_list_directories_matches_local[sessions/abc]

With this change: tests/sdk/io/ 26 passed. Downstream consumer regression: tests/sdk/conversation 708 passed.

Video/Screenshots

Side-by-side on main (same fixture, same call):

InMemory list('sessions/abc'): ['sessions/abc/def/', 'sessions/abc/event.json']  <- phantom def/
Local    list('sessions/abc'): ['sessions/abc/event.json', 'sessions/abc/nested/']
InMemory list('sessions/abc/event.json'): IndexError: list index out of range
Local    list('sessions/abc/event.json'): ['sessions/abc/event.json']

After this change the two backends return identical results for all cases in the new test suite.

InMemoryFileStore.list used bare string prefix matching, which diverged
from LocalFileStore (documented as S3-consistent) in two ways:

- listing 'a/b' also matched sibling keys like 'a/bc.txt' and invented
  phantom subdirectories from their suffixes
- listing an exact file path raised IndexError instead of returning
  [path] like LocalFileStore does

Normalize the prefix to end with '/', short-circuit exact file hits,
and skip empty suffixes. Both stores implement the same FileStore
interface and now agree on these inputs.

Signed-off-by: tomsen-ai <230283659+tomsen-ai@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant