Skip to content

fix(sandbox): keep unbounded UnixLocal workspace I/O off the event loop - #4700

Open
ayaangazali wants to merge 1 commit into
openai:mainfrom
ayaangazali:fix/sandbox-blocking-workspace-io
Open

fix(sandbox): keep unbounded UnixLocal workspace I/O off the event loop#4700
ayaangazali wants to merge 1 commit into
openai:mainfrom
ayaangazali:fix/sandbox-blocking-workspace-io

Conversation

@ayaangazali

Copy link
Copy Markdown
Contributor

Summary

UnixLocalSandboxSession does three pieces of unbounded filesystem work synchronously inside async def methods: persist_workspace tars the whole workspace, hydrate_workspace extracts it, and rm(recursive=True) removes it. All three sit on the snapshot path, so the event loop is blocked for the full duration of a workspace-sized archive or extract.

The blocking is severe enough that a resume cannot be cancelled at all. Without this change, the regression test below fails with DID NOT RAISE CancelledError: the loop never gets a chance to deliver the cancellation, so the task runs to completion instead.

Moving that work to a thread is not sufficient on its own, which is what closed #4678. asyncio.to_thread() does not stop its worker when the awaiting task is cancelled, and restore_snapshot_into_workspace_on_resume closes the archive stream in a finally as soon as its await returns:

try:
    await session.hydrate_workspace(workspace_archive)
finally:
    _close_best_effort(workspace_archive)

So a cancelled hydrate that returns early leaves a live extractor reading a closed stream and writing into a workspace that resume then clears.

This change runs the work in a worker thread and keeps waiting for that worker even while cancelled, so ownership of the archive and the workspace root is never released while something is still writing. That is the same mutation semantic the session backends already depend on: agents.memory.sqlite_session._await_mutation is used by the SQLite, SQLAlchemy, MongoDB and Redis sessions for exactly this reason, and the helper here follows it rather than inventing a second answer.

Scope is the lifecycle boundary rather than a lint rule. Only the three unbounded sites that own shared workspace state are moved. The bounded mkdir, exists and resolve calls are deliberately left alone, since being on the loop is not a demonstrated defect for them and they sit on the exec confinement path.

Tradeoff worth stating plainly: cancellation is not made prompt, only safe. A caller that cancels mid-extract still waits for the extract to finish. That is not a regression, because today the loop is blocked for that same work and cancellation cannot be delivered at all. Making cancellation prompt needs a cooperative stop inside safe_extract_tarfile and shutil.rmtree, which is a larger change and a separate one.

Test plan

tests/sandbox/test_unix_local.py::test_hydrate_workspace_cancellation_waits_for_the_extracting_worker covers the ordering that was missing: it starts a hydrate whose extractor blocks, cancels the awaiting task once the worker has actually started, and asserts that CancelledError reaches the caller only after the worker recorded completion, with the archive stream still open at that point.

Verified it fails without the source change by reverting src/: it fails with DID NOT RAISE CancelledError, which is the blocked-loop symptom itself.

.agents/skills/code-change-verification/scripts/run.sh passes end to end: format, lint, typecheck and the full suite, including all 1447 sandbox tests.

Issue number

Fixes #4675

Checks

  • I've added new tests, if relevant
  • I've run .agents/skills/code-change-verification/scripts/run.sh
  • I've confirmed all verification steps pass
  • If using Codex, I've run /review before submitting this PR

I picked this up after #4678 was closed, and I tried to build to the shape you described there rather than re-slice the same patch. The part I would most like checked is the decision to wait for the worker instead of trying to interrupt it. Waiting seemed like the only honest option given a thread cannot be stopped, and it matches what the session backends already do, but it does mean a cancel can block for as long as the extract takes. I'm a freshman in college, so if you would rather see cooperative stopping inside the tar and rmtree paths before this lands, I'm glad to take that direction instead.

persist_workspace tars the whole workspace, hydrate_workspace extracts it, and
rm(recursive=True) removes it, all synchronously inside async methods. The loop
is blocked for the full duration, so a resume cannot even be cancelled: the
cancellation never gets scheduled.

Moving that work to a thread on its own is not safe here. asyncio.to_thread()
does not stop its worker when the awaiting task is cancelled, and
restore_snapshot_into_workspace_on_resume closes the archive stream in a finally
as soon as its await returns, so a surviving extractor would read a closed
stream and write into a workspace that resume then clears.

Run the work in a worker thread and keep waiting for it even while cancelled,
matching the mutation semantics the session backends already rely on in
agents.memory.sqlite_session. Cancellation latency is unchanged, since the loop
was blocked for the same work before, while the loop stays responsive and the
archive and workspace are only released once nothing is still writing.
Copilot AI lite review requested due to automatic review settings August 26, 2026 23:56

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

UnixLocalSandbox blocks the event loop with synchronous filesystem and tar I/O

3 participants