fix(sandbox): settle PTY output before cleanup

[seratch]

This pull request fixes PTY output settlement and supersedes #4572 and #4724. PTY collectors now re-drain output at timeout boundaries, carry only complete valid UTF-8 sequences across read windows, and make bounded replacement progress for invalid E0, ED, F0, and F4 prefixes.

Terminal cleanup now follows a collector-owned settled output_closed fact across local, Docker, E2B, Cloudflare, Modal, Blaxel, and Daytona adapters, so exit visibility cannot drop queued bytes or carried suffixes.

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-08-28T10:15:42.779848Z	41be368	New commits
🔒 Security Review	✅ Completed 2026-08-28T10:15:05.801705Z	41be368	New commits

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 21c32b9985

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Codex Review: Something went wrong. Try again later by commenting “@codex review”.

An unknown error occurred

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[fscfede-beep]

I re-audited the current 41be368c head specifically around cancellation ownership. Two cancellation boundaries can still lose bytes that this PR now intends to carry across PTY windows.

[fscfede-beep]

Cancellation can drop a UTF-8 prefix that a previous window deliberately carried for this surviving session. Deterministic ordering on current head: first window ends with b"\xc3", so _drain_and_carry_incomplete_suffix() leaves [b"\xc3"] on output_chunks; the next collection drains that lead byte into local output, reaches this output_notify wait, and is cancelled. There is no outer cancellation handler, so the deque is now empty. If b"\xa9" arrives later, the next/final collection sees the continuation alone and returns � instead of é.

This is in-scope for this PR: before the new cross-window carry, a cancelled call could abandon its own drained window, but it could not consume persistent UTF-8 state intentionally handed forward by an earlier successful call and thereby corrupt later output.

The narrow fix is to make the drained local buffer transactional. Wrap collection/settlement in try/except asyncio.CancelledError; if output is non-empty, restore it synchronously to the front of output_chunks before re-raising (or keep ownership on the entry until commit). appendleft(bytes(output)) preserves these older bytes ahead of anything a producer queued while the call was running. Regression: cancel the second window while it is waiting here after draining the requeued b"\xc3"; assert the lead is back in the deque and a later b"\xa9" completes é.

[fscfede-beep]

I turned this boundary into an executable control-flow reference against the 41be368c ownership shape. The current-shape repro drains a carried b"\xc3", cancels while the next window is waiting, and deterministically leaves the deque empty; the fixed reference restores the drained local buffer synchronously with appendleft(bytes(output)), then a later b"\xa9" reconstructs é. The focused reference suite also covers the Modal transfer below: 6/6 PASS (3 tests prove current bad behavior, 3 prove the proposed ownership repairs).

The narrow shared-collector patch contract is to put the existing collection + final settlement/drain/carry phase under try/except asyncio.CancelledError; if local output is non-empty, synchronously return it to the front of output_chunks before re-raising. The handler needs to cover cancellation from poll_output, waits, settle_output, and the final drain/carry lock acquisition. I have not run the upstream suite for this patch and am not opening a competing PR.

[fscfede-beep]

Modal has the same survivor invariant one layer earlier: a successful stream read is consumed from the provider before poll_output() commits it to entry.output_chunks. Example on this branch: a previous window has carried b"\xc3" in entry.output_chunks; this call reads the continuation b"\xa9" from stdout, then is cancelled during this stderr await. The continuation lives only in local chunks, so it is lost while the older lead byte remains durable. A later call can then pair that lead with unrelated output or eventually replace it.

The exit drain has the same shape internally: _drain_modal_stream() accumulates consumed stream items in its own local bytearray across further awaits before returning them to poll_output(). Catching cancellation only in the outer shared collector therefore cannot recover bytes already removed from Modal's stream.

Please make each Modal ownership transfer cancellation-safe: either commit every successfully consumed stream chunk to entry-owned state before the next await, or catch CancelledError in both poll_output() and _drain_modal_stream() and synchronously restore their local consumed bytes to entry.output_chunks before propagating cancellation. Avoid an awaited lock acquisition as the only copy's next step unless cancellation around that acquisition also restores it. A regression should consume the UTF-8 continuation from stdout, block the following stderr read, cancel there, and verify the continuation remains entry-owned and the next collection reconstructs é.

[fscfede-beep]

I also exercised this Modal ownership transfer independently. Current-shape repro: stdout consumes the continuation b"\xa9", stderr blocks, caller cancellation arrives, and the continuation never reaches entry.output_chunks; a separate exit-drain repro consumes b"tail", cancels on the next read, and loses that local buffer too. Both are deterministic.

The fixed reference passes by (1) moving each successful stdout/stderr read into local chunks immediately before the next await, (2) catching CancelledError in poll_output and synchronously returning any locally owned chunks to entry.output_chunks, clearing local ownership immediately after a normal queue commit to avoid replay, and (3) giving _drain_modal_stream the same cancellation-return rule for its bytearray. Together with the shared collector repair, the focused reference suite is 6/6 PASS. No provider integration execution or upstream patch application is claimed.

[fscfede-beep]

Cross-link for collision/maintainer coordination: #4745 was opened later against the same PTY cross-window UTF-8 / settlement area and has since accumulated a different implementation plus several cancellation/finalization fixes. I’m reviewing both rather than opening a third implementation. The two current cancellation findings I left on this PR (3885258586, 3885258589) are specific to this branch’s carry/Modal ownership boundaries; #4745 currently avoids them with different transaction state, but has had its own finalization lifecycle issues. Recommend treating #4738 and #4745 as competing solutions and selecting one architecture rather than merging them independently.

[fscfede-beep]

Selective salvage after comparing #4745 against current 41be368c: I do not think its whole patch should be ported here. Two of the three potentially useful pieces are already covered or architecture-specific: restricted E0/ED/F0/F4 prefix handling is already tested here, and #4745's source_text + final tail truncation repair addresses a post-collection tail path this branch does not use.

One test idea is still useful here: a controlled Unix lifecycle regression where process.returncode is already visible but the output pump still owns the continuation of a split UTF-8 character. Hold the pump before it appends the continuation, assert the first collection/finalization keeps the session live and retains the lead byte, release the pump, then assert the next update returns é and closes. The current implementation looks correct by inspection because _watch_process_exit sets output_closed only after gather(*pump_tasks), but tests/sandbox/test_unix_local.py does not currently pin that ordering.

I would add that regression only after/alongside the two cancellation ownership fixes already posted in 3885258586 and 3885258589. No third competing PR from me.

[HuzaifaChaudary]

hi @seratch. i had a pull request in this area, #4745, which i have just closed in favour of this one. it predates mine and covers the same ground, so there is nothing to weigh up there.

@fscfede-beep reviewed both and suggested one thing from mine might still be worth having, so i am leaving it here as an idea rather than a patch. take it or ignore it.

this pr has the producer drained output_closed design already. the test below pins the ordering it protects, a process reaped while a pump still owns the continuation, which is the case where finalising on the exit code alone silently turns é into �:

entry = _UnixPtyProcessEntry(process=process, tty=True)   # returncode 0
# output_closed deliberately NOT set, the pump still holds the continuation
entry.output_chunks.append("é".encode()[:1])

# first update must leave the session alive with the lead byte still queued
assert first.process_id == 1
assert first.exit_code is None
assert list(entry.output_chunks) == ["é".encode()[:1]]

# then the pump delivers and closes
entry.output_chunks.append("é".encode()[1:])
entry.output_closed.set()

assert final.output.decode("utf-8") == "é"
assert final.process_id is None

what made me think it earns its place is that it fails for the right reason. swapping the predicate back to entry.process.returncode on its own gives:

PtyExecUpdate(process_id=None, output=b'\xef\xbf\xbd', exit_code=0, original_token_count=None)

so it catches a future tidy up that undoes the predicate, which is a one line change that reads like cleanup. the full version is in the closed branch at HuzaifaChaudary/openai-agents-python@78b5c4d, tests/sandbox/test_unix_local.py::TestUnixLocalPty::test_session_is_not_finalized_while_a_pump_still_holds_output.

happy to open it as a small test only pull request against this branch if you want it, or to leave it entirely. no need to reply if not.

[HuzaifaChaudary]

@seratch, short follow up to my note above and then i will stop.

the two cancellation findings @fscfede-beep raised on this pr are the same two i hit and fixed on the branch i closed, so there is working tested code for both if it saves you deriving them again. commits are on HuzaifaChaudary/openai-agents-python, against my own shape rather than yours, so they are a reference not a patch.

shared collector, 2070572e. exactly the contract he describes:

    except asyncio.CancelledError:
        if output:
            output_chunks.appendleft(bytes(output))
        raise

modal, bf682c80. the whole buffer goes to entry owned state, carried tail and new reads together, with the try covering both stream reads, the exit poll, the sleep, the drains and the final poll. assigning rather than appending is what stops the carried tail replaying, since chunks already starts with it.

each has a regression that fails on the commit before it. the modal one consumes the continuation off the stream, cancels on the next await, and asserts both halves are still entry owned.

one thing worth flagging, because it bit me right after i fixed these. once the tail flush moved in front of the registry pop, cancelling during the drain skipped _terminate_pty_entry entirely, and since the entry was already out of the map nothing could clean it up later. on blaxel that leaves a websocket and an aiohttp.ClientSession open. 78b5c4d0 puts the drain in a try with the terminate in the finally. worth checking whichever order this branch lands on.

that plus the blocked pump test above is everything i have. no reply needed, and i am not opening anything here.