fix(agentic): settle DAG state after wire drain / 网络排空后收敛 DAG 状态

[YAMY1234]

Summary / 摘要

• Preserve logical timer ownership when DAG idle-gap capping replaces a physical timer.

• Track scheduler-pending snapshot and continuation children so cancellation rolls their branch and join state back consistently.

• After duration-bounded sending stops and every frozen wire request has returned, settle residual scheduling-only DAG state that can no longer issue a legal request.

• 在 DAG idle-gap cap 替换物理 timer 时保留逻辑 timer 的 ownership。

• 跟踪仍在 scheduler 中等待的 snapshot 与 continuation child，使取消时能一致回滚 branch 与 join 状态。

• 对受 duration 限制的运行，仅在停止发送且所有冻结的网络请求均已返回后，收敛已经不可能再发出合法请求的残余调度状态。

Why / 原因

A duration-bounded DAG replay could finish every accepted HTTP request but remain blocked before phase completion because scheduler-only child and join records still made the DAG appear live. Waiting for the grace timeout is not a valid completion path: it either wastes the configured tail or turns an otherwise clean run into a timeout/cancellation result.

受 duration 限制的 DAG replay 可能已经完成所有被接收的 HTTP 请求，却仍因 scheduler-only child 与 join 记录让 DAG 看起来处于活动状态，从而无法完成 phase。依赖 grace timeout 不是合法的完成路径：它要么浪费整个尾段等待时间，要么把本来干净的运行变成超时或取消结果。

The terminal rule is intentionally fail closed. It does not truncate live wire work and does not shorten the normal drain. It runs only after sending has stopped and the frozen in-flight wire set is empty.

终态规则保持 fail-closed：不会截断仍在网络上的请求，也不会缩短正常 drain；只有在停止发送且冻结的在途网络请求集合为空后才会执行。

Validation / 验证

• Full changed-file pre-commit passed on the current upstream master base.

• 253 focused unit tests passed.

• One explicit phase-runner terminal race test passed.

• Five DAG integration cases passed.

• A full 3,600-second duration-bounded AgentX replay completed 1,310/1,310 profiling records with zero errors or cancellations; terminal stats reported no grace timeout and no phase cancellation.

• 在当前 upstream master 基线上，所有变更文件的 pre-commit 均通过。

• 253 个定向单元测试通过。

• 1 个显式 phase-runner 终态竞态测试通过。

• 5 个 DAG 集成测试通过。

• 一次完整的 3,600 秒 AgentX replay 完成 1,310/1,310 条 profiling 记录，错误和取消均为 0；终态统计未触发 grace timeout，phase 也未被取消。

---

Note

Medium Risk
Touches phase lifecycle, DAG join/dispatch teardown, and replay timer identity—areas where ordering bugs could truncate live work or leave phases stuck; changes are gated on sending-complete plus empty in-flight wire set and are heavily tested.

Overview
Fixes duration-bounded DAG/agentic replays that could finish all HTTP work yet still block phase completion because scheduler-only child timers and join bookkeeping kept has_pending_branch_work() true until the grace timeout.

LoopScheduler now keeps a stable logical handle ID when idle-gap capping reschedules a timer, so owners can still cancel the same logical replay timer after a clock jump.

BranchOrchestrator tracks scheduler-pending SPAWN turn-0 dispatches and rolls back join/descendant state via cancel_pending_delayed_dispatches() before the phase sweeps pending timers. After sending has stopped and frozen wire requests are drained, truncate_pending_after_wire_drain() clears residual join-only DAG accounting that can no longer issue a request (updates children_truncated / joins_suppressed).

AgenticReplayStrategy routes delayed child continuations through tracked handles and cancel_pending_child_turns() at the send boundary so never-started turns notify the orchestrator instead of leaving phantom joins.

PhaseRunner and CreditCallbackHandler invoke those hooks in order (orchestrator/strategy cancellation → scheduler sweep → terminal truncate) so phases can complete without waiting on grace. Docs in dag.md describe duration-boundary behavior for --request-count / cancellation.

^{Reviewed by Cursor Bugbot for commit 7ee1881. Bugbot is set up for automated code reviews on this repo. Configure here.}

[github-actions]

Try out this PR

Quick install:

pip install --upgrade --force-reinstall git+https://github.com/ai-dynamo/aiperf.git@7ee18816f7d1101a13f0518c68ce267159ee7a62

Recommended with virtual environment (using uv):

uv venv --python 3.12 && source .venv/bin/activate
uv pip install --upgrade --force-reinstall git+https://github.com/ai-dynamo/aiperf.git@7ee18816f7d1101a13f0518c68ce267159ee7a62

---

Last updated for commit: 7ee1881 • Browse code

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 2 potential issues.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

Want higher recall? High effort reviews run extra passes and find more bugs. A team admin can switch effort levels in the Cursor dashboard.

^{Reviewed by Cursor Bugbot for commit 7ee1881. Configure here.}

[cursor]

Terminal drain drops legal child work

High Severity

The new sending-boundary cancellations and truncate_pending_after_wire_drain treat is_sending_complete as a hard stop, but DAG children can still legally send after that flag flips for --num-conversations. Delayed first-turns and continuations get rolled back or truncated, so a run can finish successfully with an incomplete tree.

Additional Locations (2)

• src/aiperf/timing/phase/runner.py#L1024-L1037
• src/aiperf/timing/phase/runner.py#L1090-L1098

 

^{Reviewed by Cursor Bugbot for commit 7ee1881. Configure here.}

[cursor]

Rescheduled timer IDs can collide

Medium Severity

Idle-gap rescheduling now keeps the original id(handle) as a logical _handles key after the physical timer is replaced. That address can be reused by a later schedule_later handle, so two live timers can share one key and steal each other's cancel or fire bookkeeping.

Additional Locations (2)

• src/aiperf/common/loop_scheduler.py#L121-L126
• src/aiperf/common/loop_scheduler.py#L402-L408

 

^{Reviewed by Cursor Bugbot for commit 7ee1881. Configure here.}