feat(storage): paginate remote storage listings with a "Load more" button

[Light2Dark]

📝 Summary

Split out from #9708. Part of the work for #9662.

Adds pagination to remote storage listings:

• Threads a page_token through the storage backends, the StorageListEntriesCommand, and the StorageEntriesNotification. list_entries now returns a StorageListResult (entries + next_page_token + may_have_more).
• Surfaces a "Load more" button in the storage inspector that fetches and appends the next page, plus a "More files may exist" hint when a provider truncates a listing it cannot page through.

Screen.Recording.2026-06-10.at.11.56.00.AM.mov

📋 Pre-Review Checklist

• For large changes, or changes that affect the public API: this change was discussed or approved through an issue, on Discord, or the community discussions (Please provide a link if applicable).
• Any AI generated code has been reviewed line-by-line by the human PR author, who stands by it.
• Video or media evidence is provided for any visual changes (optional).

✅ Merge Checklist

• I have read the contributor guidelines.
• Documentation has been updated where applicable, including docstrings for API changes.
• Tests have been added for the changes made.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
marimo-docs	Ready	Preview, Comment	Jun 10, 2026 3:56am

[cubic-dev-ai]

2 issues found across 15 files

Architecture diagram

sequenceDiagram
    participant User
    participant UI as Storage Inspector (React)
    participant State as Jotai State (Atom/Reducer)
    participant Backend as Runtime Commands
    participant Storage as Storage Backend (Obstore/Fsspec)
    participant External as Object Store (S3/GCS/...)

    Note over UI,External: Paginated storage listing flow (NEW)

    UI->>State: useStorageEntries(namespace, prefix)
    State-->>UI: cached entries & pageMetadata (if any)
    alt Cache miss (first fetch)
        UI->>Backend: ListStorageEntries (pageToken: null)
        Backend->>Storage: list_entries(limit=150, page_token=None)
        Storage->>External: list entries (e.g., list_with_delimiter)
        External-->>Storage: raw entries (may be provider-truncated)
        alt Obstore: result length >= 1000
            Storage->>Storage: set may_have_more=true (provider limit)
        end
        Storage-->>Backend: StorageListResult(entries, next_page_token, may_have_more)
        Backend-->>UI: StorageEntriesNotification(entries, next_page_token, may_have_more)
        UI->>State: setEntries({entries, nextPageToken, mayHaveMore, append: false})
        State-->>UI: entriesByPath + pageMetadataByPath updated
    end
    UI->>UI: Render entries + "Load more" button (if next_page_token != null)
    UI->>UI: Render "More files may exist" hint (if may_have_more && !next_page_token)

    User->>UI: Click "Load more"
    UI->>UI: loadMore() – guard: skip if isLoadingMoreRef or no token
    UI->>Backend: ListStorageEntries (pageToken: next_page_token)
    Backend->>Storage: list_entries(limit=150, page_token)
    Storage->>Storage: slice entries[offset:offset+limit]
    Storage-->>Backend: StorageListResult(new_entries, new_next_page_token, may_have_more)
    Backend-->>UI: StorageEntriesNotification(new_entries, new_next_page_token, ...)
    UI->>State: setEntries({append: true, entries: new_entries, nextPageToken, mayHaveMore})
    State-->>UI: entries appended, pageMetadata updated
    UI->>UI: Re-render button (hidden if new_next_page_token null)
    UI->>UI: Show "More files may exist" if applicable

    opt Error during loadMore
        UI->>UI: setLoadMoreError(error)
        UI->>UI: Display error message next to button
    end

Loading

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[cubic-dev-ai]

1 issue found across 3 files (changes from recent commits).

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[Copilot]

Pull request overview

Adds end-to-end pagination support for remote storage listings, enabling the frontend storage inspector to fetch additional pages of entries via a “Load more” interaction while also signaling when a provider may have truncated results that cannot be paged further.

Changes:

• Introduces StorageListResult (entries + next_page_token + may_have_more) and threads page_token through backend implementations, runtime command handling, and notifications.
• Updates the frontend storage state/hook layer to track pagination metadata per namespace/prefix and append subsequent pages into the cached listing.
• Adds UI affordances in the storage inspector for “Load more” and a “More files may exist” hint, with corresponding test updates.

Reviewed changes

Copilot reviewed 15 out of 15 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
tests/_runtime/test_runtime_external_storage.py	Updates runtime storage listing tests to exercise page_token and next_page_token.
tests/_data/_external_storage/test_storage_models.py	Updates backend model tests for StorageListResult and adds pagination/may-have-more coverage.
packages/openapi/src/api.ts	Extends generated OpenAPI TS types for new pagination fields.
packages/openapi/api.yaml	Extends OpenAPI schema for pageToken request field and pagination fields on notifications.
marimo/_server/models/models.py	Threads page_token from HTTP request model into the runtime command.
marimo/_runtime/commands.py	Adds page_token to StorageListEntriesCommand.
marimo/_runtime/callbacks/external_storage.py	Calls backend list_entries(..., page_token=...) and includes pagination metadata in StorageEntriesNotification.
marimo/_messaging/notification.py	Adds next_page_token and may_have_more to StorageEntriesNotification.
marimo/_data/_external_storage/storage.py	Changes storage backends to return StorageListResult and implements page slicing + “may have more” detection for obstore listings.
marimo/_data/_external_storage/models.py	Introduces StorageListResult and updates the StorageBackend.list_entries contract accordingly.
frontend/src/core/storage/types.ts	Extends storage state types with per-path pagination metadata.
frontend/src/core/storage/state.ts	Implements page fetching/append logic and exposes loadMore and pagination state from useStorageEntries.
frontend/src/core/storage/tests/useStorageEntries.test.tsx	Adds coverage for load-more behavior, empty-string page tokens, and duplicate load-more suppression.
frontend/src/core/storage/tests/state.test.ts	Adds reducer/state coverage for pagination metadata + append behavior.
frontend/src/components/storage/storage-inspector.tsx	Adds “Load more” button and “More files may exist” hint UI using new hook outputs.