diff --git a/CHANGELOG.md b/CHANGELOG.md index 9252f5a..a18ab05 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -3,8 +3,19 @@ ## Unreleased - Expand queued prompt templates and Agent Skills at delivery, with arguments, images, short aliases such as `/bro`, and full-batch restoration if expansion fails. +- Make command rows compaction-aware: idle `/compact` uses Pi's public compaction API, `/reload` waits for direct or automatic compaction to settle, and ordinary messages stay in Pi's native compaction queue. +- Preserve stable row IDs, lanes, attachments and pause state when committed rows cross a `/reload` runtime swap, including rows added after reload scheduling and repeated queued reloads. +- Keep image-bearing command text as a normal queued message so attachments are never discarded. +- Hold queued follow-ups while Pi decides whether an error, length stop or context overflow needs retry or automatic compaction. +- Restore and pause `/compact` when compaction cannot start, and restore only an unsent all-mode tail after a synchronous partial handoff failure. +- Leave RPC, JSON and print-mode input unchanged; queue ownership is TUI-only. +- Rebind editor guards across runtime reloads and capture command rows even while slash autocomplete is visible. +- Normalize native post-compaction input classification so whitespace and immediate hidden built-ins cannot strand queued rows. +- Add deterministic AgentSession retry coverage and a reproducible real-TUI evidence harness for manual/overflow compaction, abort recovery, native ordering, repeated reloads, resources and all-mode delivery. +- Keep Pi package ranges unpinned so compatibility validation follows current Pi releases. + - Add command rows: `/compact [instructions]` and `/reload` queue in FIFO position and execute only once the agent is idle, so rows behind them wait — e.g. a queued `continue` delivers after compaction completes. -- Queue a mid-run `Enter` on `/reload` instead of surfacing Pi's built-in "wait until the agent finishes" warning; mid-run `Enter` on `/compact` keeps Pi's built-in behaviour. +- Queue a mid-run `Enter` on `/reload` instead of surfacing Pi's built-in "wait until the agent finishes" warning; mid-run `Enter` on `/compact` uses Pi's public compaction API and holds queued rows until compaction settles. - Restore rows queued behind a `/reload` after the runtime swap. - Execute idle `Option+Enter` command submissions instead of letting them reach the LLM as text. diff --git a/README.md b/README.md index 5871bdf..1ea0089 100644 --- a/README.md +++ b/README.md @@ -67,7 +67,7 @@ The extension keeps Pi’s 2 delivery classes: - each lane keeps its own first-in, first-out order - Pi’s `one-at-a-time` and `all` settings apply independently at active-run delivery boundaries -The extension hands messages back to Pi’s native queues only when their delivery boundary arrives. They remain visible and editable before that point. Pi records delivered rows as normal user messages. +The extension hands messages back to Pi’s native queues only when their delivery boundary arrives. They remain visible and editable before that point. Pi records delivered rows as normal user messages. Queue ownership is TUI-only; RPC, JSON and print-mode input pass through unchanged. ## Prompt templates and Agent Skills @@ -77,12 +77,15 @@ Pi cannot invoke arbitrary commands through its public extension API. `/compact` ## Command rows -Rows whose text is exactly `/compact`, `/compact ` or `/reload` are command rows. They execute the Pi command instead of becoming an LLM message: +Text-only rows whose text is exactly `/compact`, `/compact ` or `/reload` are command rows. A row with image attachments remains a normal message even if its text matches a command, so attachments are never discarded. Command rows execute the Pi command instead of becoming an LLM message: - `Option+Enter` while the agent works queues the command in follow-up order - a command row executes only once the agent is idle; rows behind it wait — so `/compact` followed by `continue` compacts first and delivers `continue` after compaction completes -- `/reload` runs Pi’s built-in reload; rows queued behind it are restored after the runtime swap -- `Enter` on `/reload` while the agent works queues it too, replacing Pi’s built-in “wait until the agent finishes” warning; `Enter` on `/compact` keeps Pi’s built-in immediate behaviour +- `/reload` runs Pi’s built-in reload; committed rows queued behind it retain their IDs, lanes, attachments and pause state across the runtime swap +- idle `/compact` uses Pi’s public compaction API so queued rows resume when compaction finishes; a start failure restores and pauses the command row +- `/reload` submitted while the agent works or tracked compaction runs stays queued instead of showing Pi’s built-in wait warning +- `Enter` on `/compact` while the agent works uses Pi’s public compaction API and holds visible rows until compaction settles +- ordinary messages submitted during compaction remain in Pi’s native queue - `Option+Enter` on a command while the agent is idle executes it immediately instead of sending the text to the model - command rows show a `⚙` marker and pause, resume and edit like any other row; editing a row into or out of command form just works @@ -106,13 +109,17 @@ A touched head row is pinned until you save or cancel. In `one-at-a-time` mode, Aborting a run pauses both visible lanes. This prevents a follow-up from starting immediately after the abort. -Press `Enter` on the empty composer to resume. A failed handoff returns the affected batch to the front of its lane. +Press `Enter` on the empty composer to resume. A synchronous handoff or preflight failure returns the affected batch to the front of its lane. -Queue state, pause state and edit drafts are session-local. They never enter the Pi transcript. +Queue state, pause state and edit drafts are session-local. They never enter the Pi transcript or persistent session data. A `/reload` runtime swap carries committed rows and pause state through a short in-process handoff; unsaved edit drafts do not cross the swap. -## Proof limitation +## Public API limits -If an `all`-mode lane stays pinned until the agent settles, saving from idle restarts the run with that lane’s head. Pi receives the remaining rows at the next native boundary. Exact single-batch restart after this edge case remains open before release. +Pi’s public `sendUserMessage` API is fire-and-forget. The extension restores synchronous dispatch failures and preflight/expansion failures without reordering, but Pi does not expose later asynchronous input rejection to extensions. Inferring rejection from queue timing could duplicate a delayed successful handoff, so the extension does not do that. + +Pi also exposes queued `/reload` only through the TUI editor’s `void` submit callback. The extension prevents known busy and compaction conflicts and restores trailing rows on a successful runtime swap, but Pi cannot acknowledge or reject that submit back to the extension. + +If an `all`-mode lane stays pinned until the agent settles, saving from idle starts the new run with the lane head, then delivers the remaining rows in FIFO order at the next native boundary. The public API has no atomic idle-to-native-queue batch operation, so this restart cannot be one native batch. ## Editor composition @@ -127,12 +134,13 @@ The extension composes with custom editors including raw-paste and pi-session-hu ```bash npm install npm run ci +./test/tui-evidence.sh /tmp/pi-queue-tui-evidence pi -e ./index.ts ``` -The automated suite covers delivery, editing, command rows, resource expansion, recovery, images and editor composition. Check TUI changes in a real Pi session as well. +The automated suite covers delivery, editing, command rows, resource expansion, recovery, images, editor composition, repeated reloads, real retry ordering, real manual compaction success/failure and real automatic overflow compaction. The tmux harness exercises the same paths through Pi's real TUI, including actual runtime reloads and native post-compaction input. -Automated against Pi 0.80.9 and smoke-tested interactively with Pi 0.84.1. +The Pi package ranges are intentionally unpinned. The full suite and real-TUI harness are verified against the current resolved Pi release; see [the validation record](docs/validation.md) for exact commands and evidence. ## Security diff --git a/docs/validation.md b/docs/validation.md new file mode 100644 index 0000000..8f691e6 --- /dev/null +++ b/docs/validation.md @@ -0,0 +1,93 @@ +# Compaction and reload validation + +This document records the deterministic validation matrix for compaction-aware command rows. The implementation remains extension-only and uses public Pi extension APIs. + +## Automated suite + +The Pi package ranges are intentionally unpinned. The lockfile records the versions used for a reproducible checkout, but the package manifest does not declare an artificial Pi compatibility target. + +Run the resolved dependency set: + +```bash +npm ci --ignore-scripts +npm run ci +``` + +Refresh to the current Pi packages before compatibility review: + +```bash +npm update --ignore-scripts \ + @earendil-works/pi-ai \ + @earendil-works/pi-coding-agent \ + @earendil-works/pi-tui +npm run ci +``` + +The suite covers queue/edit invariants, command classification, images, one-at-a-time and all-mode delivery, synchronous partial handoff restoration, non-TUI pass-through, prompt and Skill expansion, manual compaction success/failure, automatic overflow compaction, retry ordering, repeated reload restoration, and compaction/native-input ordering. + +Latest result with Pi 0.84.1: 81 tests passed. + +## Real TUI evidence + +`test/tui-evidence.sh` starts the real Pi 0.84.1 TUI under tmux with a deterministic faux provider. It uses actual terminal key sequences, public compaction lifecycle events, public provider registration, actual runtime reloads, and Pi's real native compaction queue. + +Run: + +```bash +./test/tui-evidence.sh /tmp/pi-queue-tui-evidence +``` + +The output directory contains plain terminal captures, provider-call logs, lifecycle-event logs, and runtime-initialization logs. Run it immediately before review so `summary.txt` records the exact Pi version, commit and working-tree state under test. A release evidence run should report `working tree: clean`. + +The latest complete run reported: + +```text +pi: 0.84.1 +commit: +working tree: clean +manual events: {"event":"session_before_compact","reason":"manual"} {"event":"session_before_compact","reason":"manual"} +overflow events: {"event":"session_before_compact","reason":"overflow"} {"event":"session_before_compact","reason":"threshold"} +runtime initializations across two queued reloads: 3 +captures: abort-paused, manual-reload-resources, native-before-command, automatic-overflow, all-mode +``` + +The three runtime initializations are the initial load plus two queued `/reload` rows. The final queued message ran after both reloads. + +The semantic capture excerpts were: + +```text +[compaction] +Compacted from 798 tokens +FAUX RESPONSE: after manual compaction + +Error: Compaction failed: Summarization failed: synthetic TUI summary failure +FAUX RESPONSE: after failed compaction + +Operation aborted +follow-ups (1) · paused +enter resume · option+up edit · escape keep paused +FAUX RESPONSE: after abort resume + +Reloaded keybindings, extensions, skills, prompts, themes, and context files +FAUX RESPONSE: after repeated reload + +PROMPT EXPANDED: first=alpha all=alpha beta default=fallback +[skill] bro +FAUX RESPONSE: +``` + +The native post-compaction ordering capture showed the ordinary native message finishing before the extension-owned command, and `/reload` never reached the model: + +```text +[compaction] +Compacted from 785 tokens +ordinary native during compaction +FAUX RESPONSE: ordinary native during compaction +Reloaded keybindings, extensions, skills, prompts, themes, and context files +``` + +The overflow event log recorded `reason: "overflow"`, the TUI rendered a compaction entry, and `overflow-provider-calls.jsonl` proved the queued follow-up completed exactly once. `all-mode-provider-calls.jsonl` proved all three rows reached Pi exactly once in FIFO order; the all-mode capture rendered them together before the final response. + +## Public API boundary + +`ExtensionAPI.sendUserMessage` and the TUI editor submit callback return `void`. The extension can restore synchronous handoff failures and preflight/expansion failures, but it cannot prove every later asynchronous acceptance or rejection without risking duplicate delivery. Queued `/reload` likewise has no result channel. These limits are documented in the README and are not hidden by timing heuristics. diff --git a/index.ts b/index.ts index dabcaaa..d855803 100644 --- a/index.ts +++ b/index.ts @@ -1,4 +1,5 @@ import type { ImageContent, TextContent } from "@earendil-works/pi-ai"; +import { isContextOverflow } from "@earendil-works/pi-ai/compat"; import { CustomEditor, keyText, @@ -10,7 +11,7 @@ import { } from "@earendil-works/pi-coding-agent"; import { matchesKey, truncateToWidth, visibleWidth, type Component, type EditorComponent } from "@earendil-works/pi-tui"; import { extractInlineEditorLines } from "./editor-render.ts"; -import { expandQueuedInput } from "./queued-input.ts"; +import { expandQueuedInput, queuesDuringCompaction } from "./queued-input.ts"; import { DeliveryQueue, parseQueuedCommand, @@ -24,20 +25,16 @@ const WIDGET_ID = "queue-steer.timeline"; const EDITOR_FEATURES = Symbol.for("@tmustier/pi-editor-features"); const QUEUE_STEER_FEATURE = "queue-steer"; const NEXT_ROW_KEY = "alt+down"; -const RELOAD_STASH_KEY = "@tmustier/pi-queue-steer.reload-stash"; const SUBMIT_GUARD = Symbol.for("@tmustier/pi-queue-steer.submit-guard"); -/** Rows surviving a queued /reload, parked on globalThis across the runtime swap. */ -interface ReloadStashRow { - lane: QueueLane; - text: string; - images: ImageContent[]; -} +/** Queue state parked on globalThis across Pi's in-process runtime swap. */ interface ReloadStash { - at: number; - rows: ReloadStashRow[]; + paused: boolean; + rows: QueuedMessage[]; +} +declare global { + var __tmustierPiQueueSteerReloadStash: ReloadStash | undefined; } -const globalStore = globalThis as unknown as Record; const REMOVE_ROW_KEY = "alt+x"; const TOGGLE_LANE_KEY = "alt+t"; @@ -232,18 +229,37 @@ function userContent(item: QueuedMessage): string | (TextContent | return [{ type: "text", text: item.text }, ...item.images]; } +function itemCommand(item: Pick, "text" | "images">): QueuedCommand | undefined { + // Treat an image-bearing row as a message so executing a command can never + // silently discard its attachments. + return item.images.length === 0 ? parseQueuedCommand(item.text) : undefined; +} + export default function queueSteerExtension(pi: ExtensionAPI) { const queue = new DeliveryQueue(); let editSession: QueueEditSession | undefined; let activeContext: ExtensionContext | undefined; let renderInlineEditor: InlineEditorRenderer | undefined; let editorInstallTimer: ReturnType | undefined; + let baseEditorFactory: EditorFactory | undefined; + let baseEditorFactoryCaptured = false; + let reloadSubmitTimer: ReturnType | undefined; let renderingInline = false; let paused = false; let settingsManager: SettingsManager | undefined; - // True while a queued /compact (or a just-dispatched /reload) is executing; - // suspends all lane dispatch until the command completes. - let commandRunning = false; + let blockingActivity: "compact" | "auto-compact" | "reload" | undefined; + let compactionFinishTimer: ReturnType | undefined; + let nativeCompactionInputQueued = false; + let nativeCompactionTurnStarted = false; + const isCompacting = (): boolean => blockingActivity === "compact" || blockingActivity === "auto-compact"; + const trackNativeCompactionSubmission = ( + text: string, + behavior: "submit" | "followUp" = "submit", + ): void => { + if (isCompacting() && queuesDuringCompaction(text, pi.getCommands(), behavior)) { + nativeCompactionInputQueued = true; + } + }; // Pi's own editor submit handler, captured by the submit guard. Replaying text // through it is the only public route to the built-in /reload. let tuiSubmit: ((text: string) => void) | undefined; @@ -291,15 +307,16 @@ export default function queueSteerExtension(pi: ExtensionAPI) { const decorated = queue.snapshot().map((item): TimelineItem => { const lane = editSession?.laneFor(item.id) ?? item.lane; const text = editSession?.textFor(item.id) ?? item.text; + const images = editSession?.imagesFor(item.id) ?? item.images; return { ...item, text, - images: editSession?.imagesFor(item.id) ?? item.images, + images, lane, removed: editSession?.isRemoved(item.id) ?? false, movedLane: lane !== item.lane, held: heldLane[item.lane] && (modes[item.lane] === "all" || heads[item.lane] === item.id), - command: parseQueuedCommand(text), + command: itemCommand({ text, images }), }; }); return [ @@ -336,8 +353,8 @@ export default function queueSteerExtension(pi: ExtensionAPI) { // A command row at the lane head holds everything behind it (FIFO) until the // agent settles and dispatchFromIdle executes it. const takeLaneBatch = (lane: QueueLane): QueuedMessage[] => { - if (paused || commandRunning || queue.laneLength(lane) === 0 || laneIsHeld(lane)) return []; - const isMessage = (item: QueuedMessage) => parseQueuedCommand(item.text) === undefined; + if (paused || blockingActivity || queue.laneLength(lane) === 0 || laneIsHeld(lane)) return []; + const isMessage = (item: QueuedMessage) => itemCommand(item) === undefined; if (queueModes()[lane] === "all") return queue.shiftWhile(lane, isMessage); const head = queue.peek(lane); if (!head || !isMessage(head)) return []; @@ -360,23 +377,19 @@ export default function queueSteerExtension(pi: ExtensionAPI) { pauseAfterPreparationFailure(ctx, lane, error); return false; } - const pendingBefore = ctx.hasPendingMessages(); renderQueue(ctx); + let submitted = 0; try { for (const item of prepared) { pi.sendUserMessage(userContent(item), { deliverAs: lane }); + submitted += 1; } - // sendUserMessage is fire-and-forget. Keep the awaited boundary - // handler open until async input preflight reaches Pi's native queue. - for (let attempt = 0; attempt < 5 && !ctx.hasPendingMessages(); attempt += 1) { - await new Promise((resolve) => setTimeout(resolve, 0)); - } - if (!pendingBefore && !ctx.hasPendingMessages()) { - throw new Error("Pi did not accept the queued message at this delivery boundary"); - } + // The public send API is fire-and-forget. Once invoked, do not infer + // rejection from aggregate queue timing: a delayed preflight could + // otherwise accept the original after we restored and duplicate it. return true; } catch (error) { - queue.prependMany(items); + queue.prependMany(items.slice(submitted)); renderQueue(ctx); ctx.ui.notify( `Could not deliver queued ${laneLabel(lane)}: ${error instanceof Error ? error.message : String(error)}`, @@ -398,50 +411,33 @@ export default function queueSteerExtension(pi: ExtensionAPI) { // Execute the command row at the lane head. Only called when the agent is idle. const executeCommandRow = (ctx: ExtensionContext, lane: QueueLane): boolean => { - const next = queue.shift(lane); + const next = queue.peek(lane); if (!next) return false; - const command = parseQueuedCommand(next.text); - if (!command) { - queue.prepend(next); + const command = itemCommand(next); + if (!command) return false; + const submit = tuiSubmit; + if (command.kind === "reload" && !submit) { + paused = true; + renderQueue(ctx); + ctx.ui.notify("Could not run queued /reload; queue paused because no interactive submit handler is available", "error"); return false; } + queue.shift(lane); paused = false; renderQueue(ctx); if (command.kind === "compact") { - commandRunning = true; - ctx.ui.notify(`Running queued /compact${command.instructions ? ` (${command.instructions})` : ""}`, "info"); - const resume = () => { - commandRunning = false; - const current = activeContext ?? ctx; - renderQueue(current); - if (!paused && !editSession && queue.length > 0 && current.isIdle()) dispatchFromIdle(current); - }; - ctx.compact({ - customInstructions: command.instructions, - onComplete: resume, - onError: (error) => { - (activeContext ?? ctx).ui.notify(`Queued /compact failed: ${error.message}`, "error"); - resume(); - }, - }); - return true; - } - const submit = tuiSubmit; - if (!submit) { - ctx.ui.notify("Queued /reload dropped: no interactive editor to run it through", "error"); + if (startCompaction(ctx, command.instructions)) return true; + queue.prepend(next); + paused = true; renderQueue(ctx); return false; } - if (queue.length > 0) { - const stash: ReloadStash = { - at: Date.now(), - rows: queue.snapshot().map((item) => ({ lane: item.lane, text: item.text, images: item.images })), - }; - globalStore[RELOAD_STASH_KEY] = stash; - } - commandRunning = true; + blockingActivity = "reload"; // Defer so the extension runtime is never torn down from inside this handler. - setTimeout(() => submit("/reload"), 0); + reloadSubmitTimer = setTimeout(() => { + reloadSubmitTimer = undefined; + submit?.("/reload"); + }, 0); return true; }; @@ -474,7 +470,7 @@ export default function queueSteerExtension(pi: ExtensionAPI) { const dispatchFromIdle = (ctx: ExtensionContext): boolean => { activeContext = ctx; - if (commandRunning) { + if (blockingActivity) { renderQueue(ctx); return false; } @@ -488,16 +484,69 @@ export default function queueSteerExtension(pi: ExtensionAPI) { return false; } const head = queue.peek(lane); - if (head && parseQueuedCommand(head.text)) return executeCommandRow(ctx, lane); + if (head && itemCommand(head)) return executeCommandRow(ctx, lane); return sendHeadMessage(ctx, lane); }; + const deferCompactionFinish = ( + ctx: ExtensionContext, + activity: "compact" | "auto-compact", + ): void => { + compactionFinishTimer = setTimeout(() => { + compactionFinishTimer = undefined; + if (blockingActivity !== activity) return; + // Pi flushes ordinary TUI submissions after compaction without + // awaiting prompt preflight. Keep command rows behind that native run. + if (nativeCompactionInputQueued) { + renderQueue(activeContext ?? ctx); + return; + } + blockingActivity = undefined; + nativeCompactionInputQueued = false; + nativeCompactionTurnStarted = false; + const current = activeContext ?? ctx; + renderQueue(current); + if (!paused && !editSession && queue.length > 0 && current.isIdle()) dispatchFromIdle(current); + }, 0); + }; + + const startCompaction = (ctx: ExtensionContext, instructions: string | undefined): boolean => { + blockingActivity = "compact"; + nativeCompactionInputQueued = false; + nativeCompactionTurnStarted = false; + try { + ctx.compact({ + customInstructions: instructions, + onComplete: () => { + if (!nativeCompactionInputQueued) deferCompactionFinish(ctx, "compact"); + }, + onError: () => { + if (!nativeCompactionInputQueued) deferCompactionFinish(ctx, "compact"); + }, + }); + return true; + } catch (error) { + blockingActivity = undefined; + ctx.ui.notify( + `Could not start compaction: ${error instanceof Error ? error.message : String(error)}`, + "error", + ); + return false; + } + }; + + const deferCommand = (ctx: ExtensionContext, text: string): void => { + queue.enqueue("followUp", text); + paused = false; + renderQueue(ctx); + }; + const sendFollowUpNow = (ctx: ExtensionContext): boolean => { const head = queue.peek("followUp"); if (!head) return false; - const headCommand = parseQueuedCommand(head.text); + const headCommand = itemCommand(head); if (headCommand) { - if (commandRunning || !ctx.isIdle()) { + if (blockingActivity === "reload" || !ctx.isIdle()) { ctx.ui.notify(`Queued /${headCommand.kind} runs when the agent is idle`, "info"); return false; } @@ -527,7 +576,7 @@ export default function queueSteerExtension(pi: ExtensionAPI) { renderQueue(ctx); // A pinned head may have let the agent settle while it was edited. - if (ctx.isIdle() && !paused) dispatchFromIdle(ctx); + if (ctx.isIdle() && !paused && !blockingActivity) dispatchFromIdle(ctx); }; const selectQueueItem = (ctx: ExtensionContext, direction: "previous" | "next"): void => { @@ -635,6 +684,17 @@ export default function queueSteerExtension(pi: ExtensionAPI) { } } + if (keybindings.matches(data, "app.message.followUp")) { + const text = (editor.getExpandedText?.() ?? editor.getText()).trim(); + if (isCompacting() && parseQueuedCommand(text)) { + deferCommand(ctx, text); + editor.addToHistory?.(text); + editor.setText(""); + return; + } + trackNativeCompactionSubmission(text, "followUp"); + } + if (queue.length > 0 && keybindings.matches(data, "app.message.dequeue")) { selectQueueItem(ctx, "previous"); return; @@ -655,6 +715,10 @@ export default function queueSteerExtension(pi: ExtensionAPI) { !editor.getText().trim() && keybindings.matches(data, "tui.input.submit") ) { + if (isCompacting()) { + ctx.ui.notify("Queued messages will run after compaction finishes", "info"); + return; + } if (paused) { paused = false; if (ctx.isIdle()) dispatchFromIdle(ctx); @@ -671,17 +735,17 @@ export default function queueSteerExtension(pi: ExtensionAPI) { return editor; }) as ComposedEditorFactory; factory[EDITOR_FEATURES] = new Set([...features, QUEUE_STEER_FEATURE]); + // Preserve the factory from before this runtime's first wrapper. A later + // unmarked composer may itself close over our wrapper; restoring that on + // reload would carry stale submit guards into the replacement runtime. + if (!baseEditorFactoryCaptured) { + baseEditorFactory = previousFactory; + baseEditorFactoryCaptured = true; + } ctx.ui.setEditorComponent(factory); renderQueue(ctx); }; - /** - * Wrap the semantic submit point (after autocomplete resolution) so a mid-run - * Enter on /reload queues it instead of hitting Pi's built-in "wait until the - * agent finishes" warning. Everything else, including mid-run /compact, - * passes through to Pi's own dispatch unchanged. The wrap also captures Pi's - * submit handler for the queued-/reload replay. - */ const installSubmitGuard = (editor: EditorComponent, ctx: ExtensionContext): void => { const guarded = editor as EditorComponent & { [SUBMIT_GUARD]?: boolean }; if (guarded[SUBMIT_GUARD]) return; @@ -690,12 +754,25 @@ export default function queueSteerExtension(pi: ExtensionAPI) { if (innerSubmit) tuiSubmit = innerSubmit; const wrappedSubmit = (text: string) => { const command = parseQueuedCommand(text); - if (command?.kind === "reload" && !editSession && (commandRunning || !ctx.isIdle())) { + if (!editSession && command && isCompacting()) { + deferCommand(ctx, text); + editor.addToHistory?.(text); + editor.setText(""); + return; + } + if (command?.kind === "compact" && !editSession) { + editor.addToHistory?.(text); + editor.setText(""); + startCompaction(ctx, command.instructions); + return; + } + if (command?.kind === "reload" && !editSession && (blockingActivity === "reload" || !ctx.isIdle())) { queue.enqueue("followUp", text, []); paused = false; renderQueue(ctx); return; } + if (!editSession) trackNativeCompactionSubmission(text); innerSubmit?.(text); }; Object.defineProperty(editor, "onSubmit", { @@ -736,7 +813,7 @@ export default function queueSteerExtension(pi: ExtensionAPI) { }); pi.on("input", (event, ctx) => { - if (event.source !== "interactive") return { action: "continue" }; + if (ctx.mode !== "tui" || event.source !== "interactive") return { action: "continue" }; activeContext = ctx; // Safety net for editor wrappers installed after ours: an editing submit @@ -746,6 +823,7 @@ export default function queueSteerExtension(pi: ExtensionAPI) { return { action: "handled" }; } + const command = parseQueuedCommand(event.text); if (event.streamingBehavior === "steer" || event.streamingBehavior === "followUp") { queue.enqueue(event.streamingBehavior, event.text, event.images); paused = false; @@ -753,24 +831,36 @@ export default function queueSteerExtension(pi: ExtensionAPI) { return { action: "handled" }; } - // Idle command submissions (e.g. alt+enter bypasses Pi's built-in dispatch) - // would otherwise reach the LLM as text. Route them through the queue; when - // nothing is running they execute immediately. - if (event.streamingBehavior === undefined && parseQueuedCommand(event.text) && (ctx.isIdle() || commandRunning)) { + // Alt+Enter can bypass Pi's built-in command dispatch while idle. + if (event.streamingBehavior === undefined && command && (event.images?.length ?? 0) === 0 && ctx.isIdle()) { queue.enqueue("followUp", event.text, event.images ?? []); paused = false; renderQueue(ctx); - if (!commandRunning && ctx.isIdle()) dispatchFromIdle(ctx); + dispatchFromIdle(ctx); return { action: "handled" }; } return { action: "continue" }; }); + pi.on("session_before_compact", (event, ctx) => { + activeContext = ctx; + if (blockingActivity || event.reason === "manual") return; + blockingActivity = "auto-compact"; + nativeCompactionInputQueued = false; + nativeCompactionTurnStarted = false; + renderQueue(ctx); + }); + + pi.on("turn_start", (_event, ctx) => { + activeContext = ctx; + if (isCompacting() && nativeCompactionInputQueued) nativeCompactionTurnStarted = true; + }); + pi.on("turn_end", async (event, ctx) => { activeContext = ctx; if (event.message.role === "assistant" && event.message.stopReason === "aborted") { - if (queue.length > 0) paused = true; + if (queue.length > 0 && blockingActivity !== "compact") paused = true; renderQueue(ctx); return; } @@ -781,9 +871,22 @@ export default function queueSteerExtension(pi: ExtensionAPI) { // Pi checks its native queues again after extension agent_end handlers. // Feeding one item (or an all-mode batch) here preserves native follow-up // continuation semantics without relinquishing later editable rows early. - pi.on("agent_end", async (_event, ctx) => { + pi.on("agent_end", async (event, ctx) => { activeContext = ctx; if (paused) return; + const lastMessage = event.messages.at(-1); + if ( + lastMessage?.role === "assistant" + && ( + lastMessage.stopReason === "length" + || lastMessage.stopReason === "error" + || isContextOverflow(lastMessage, ctx.model?.contextWindow ?? 0) + ) + ) { + // Pi decides whether to retry or auto-compact only after agent_end. + // Injecting a follow-up here would start it first and hide that signal. + return; + } if (queue.laneLength("steer") > 0) { await dispatchLaneAtBoundary(ctx, "steer"); return; @@ -793,30 +896,66 @@ export default function queueSteerExtension(pi: ExtensionAPI) { pi.on("agent_settled", (_event, ctx) => { activeContext = ctx; + if (blockingActivity === "compact" || blockingActivity === "auto-compact") { + const activity = blockingActivity; + if (nativeCompactionInputQueued && !nativeCompactionTurnStarted) { + renderQueue(ctx); + return; + } + // The ordinary post-compaction turn, if any, is now fully settled. + nativeCompactionInputQueued = false; + deferCompactionFinish(ctx, activity); + return; + } renderQueue(ctx); - if (!paused && !editSession && queue.length > 0 && ctx.isIdle()) dispatchFromIdle(ctx); + if (!paused && !editSession && queue.length > 0 && ctx.isIdle() && !blockingActivity) dispatchFromIdle(ctx); }); - pi.on("session_shutdown", () => { + pi.on("session_shutdown", (event) => { + if (event.reason === "reload" && queue.length > 0) { + const stash: ReloadStash = { paused, rows: queue.snapshot() }; + globalThis.__tmustierPiQueueSteerReloadStash = stash; + } else { + globalThis.__tmustierPiQueueSteerReloadStash = undefined; + } if (editorInstallTimer) clearTimeout(editorInstallTimer); - if (activeContext?.hasUI) activeContext.ui.setWidget(WIDGET_ID, undefined); + if (reloadSubmitTimer) clearTimeout(reloadSubmitTimer); + if (compactionFinishTimer) clearTimeout(compactionFinishTimer); + if (activeContext?.hasUI) { + const currentFactory = activeContext.ui.getEditorComponent(); + if ( + baseEditorFactoryCaptured + && currentFactory + && editorFeatures(currentFactory).has(QUEUE_STEER_FEATURE) + ) { + activeContext.ui.setEditorComponent(baseEditorFactory); + } + activeContext.ui.setWidget(WIDGET_ID, undefined); + } activeContext = undefined; renderInlineEditor = undefined; editorInstallTimer = undefined; + baseEditorFactory = undefined; + baseEditorFactoryCaptured = false; + reloadSubmitTimer = undefined; + compactionFinishTimer = undefined; editSession = undefined; settingsManager = undefined; paused = false; - commandRunning = false; + blockingActivity = undefined; + nativeCompactionInputQueued = false; + nativeCompactionTurnStarted = false; tuiSubmit = undefined; queue.clear(); }); - /** Re-adopt rows that a queued /reload parked across the runtime swap. */ + /** Re-adopt committed queue state after Pi's in-process runtime swap. */ function restoreReloadStash(reason: string, ctx: ExtensionContext): void { - const stash = globalStore[RELOAD_STASH_KEY] as ReloadStash | undefined; - delete globalStore[RELOAD_STASH_KEY]; - if (!stash || reason !== "reload" || Date.now() - stash.at > 30_000 || stash.rows.length === 0) return; - for (const row of stash.rows) queue.enqueue(row.lane, row.text, row.images); + const stash = globalThis.__tmustierPiQueueSteerReloadStash; + globalThis.__tmustierPiQueueSteerReloadStash = undefined; + if (!stash || reason !== "reload" || stash.rows.length === 0) return; + queue.restore(stash.rows); + paused = stash.paused; ctx.ui.notify(`Restored ${stash.rows.length} queued row${stash.rows.length === 1 ? "" : "s"} after reload`, "info"); setTimeout(() => { const current = activeContext; diff --git a/package-lock.json b/package-lock.json index e2d0eac..fa99496 100644 --- a/package-lock.json +++ b/package-lock.json @@ -9,9 +9,9 @@ "version": "0.1.0", "license": "MIT", "devDependencies": { - "@earendil-works/pi-ai": "0.80.9", - "@earendil-works/pi-coding-agent": "0.80.9", - "@earendil-works/pi-tui": "0.80.9", + "@earendil-works/pi-ai": "*", + "@earendil-works/pi-coding-agent": "*", + "@earendil-works/pi-tui": "*", "@types/node": "24.12.4", "tsx": "4.22.1", "typescript": "5.9.3" @@ -513,14 +513,15 @@ } }, "node_modules/@earendil-works/pi-ai": { - "version": "0.80.9", - "resolved": "https://registry.npmjs.org/@earendil-works/pi-ai/-/pi-ai-0.80.9.tgz", - "integrity": "sha512-kHsH5nO4FU7mbKnskK0BVPVuWzNb2DrZtiN1fb6LamP+6BMI8xEZiAOw2fqs4VudvlMQgOLjtbgErv+kNJRPIg==", + "version": "0.84.1", + "resolved": "https://registry.npmjs.org/@earendil-works/pi-ai/-/pi-ai-0.84.1.tgz", + "integrity": "sha512-wMsAdJMxuNri08vLqTyYVI201DQQezGhPSTkzYsHdw5dYX3rCNwEmSvpaAwhi7ELKI/2tE/CEgSWg/6iRxSgdQ==", "dev": true, "license": "MIT", "dependencies": { "@anthropic-ai/sdk": "0.91.1", "@aws-sdk/client-bedrock-runtime": "3.1048.0", + "@earendil-works/pi-telemetry": "^0.84.1", "@google/genai": "1.52.0", "@mistralai/mistralai": "2.2.6", "@opentelemetry/api": "1.9.0", @@ -529,7 +530,7 @@ "https-proxy-agent": "7.0.6", "openai": "6.26.0", "partial-json": "0.1.7", - "typebox": "1.1.38" + "typebox": "1.3.7" }, "bin": { "pi-ai": "dist/cli.js" @@ -539,21 +540,24 @@ } }, "node_modules/@earendil-works/pi-coding-agent": { - "version": "0.80.9", - "resolved": "https://registry.npmjs.org/@earendil-works/pi-coding-agent/-/pi-coding-agent-0.80.9.tgz", - "integrity": "sha512-Clgx2Bg5NbMcCpGxusSDQwE+GC0g/d6sCBluE9aypPgSgtJ6n8VmZIIT6auXObMskpRgkr+XZ77wG5hf+cSDtg==", + "version": "0.84.1", + "resolved": "https://registry.npmjs.org/@earendil-works/pi-coding-agent/-/pi-coding-agent-0.84.1.tgz", + "integrity": "sha512-ncAqFrG+iybuPGOhMiZoEHkEzTpJgz3guYD32pD+M7ucc0WeHmauP6wa7qwP8V/KWvsZDVNa5XGsdZ7fkC7w7A==", "dev": true, "hasShrinkwrap": true, "license": "MIT", "dependencies": { - "@earendil-works/pi-agent-core": "^0.80.9", - "@earendil-works/pi-ai": "^0.80.9", - "@earendil-works/pi-tui": "^0.80.9", + "@earendil-works/pi-agent-core": "^0.84.1", + "@earendil-works/pi-ai": "^0.84.1", + "@earendil-works/pi-client": "^0.84.1", + "@earendil-works/pi-protocol": "^0.84.1", + "@earendil-works/pi-tui": "^0.84.1", "@silvia-odwyer/photon-node": "0.3.4", "chalk": "5.6.2", "cross-spawn": "7.0.6", "diff": "8.0.4", "glob": "13.0.6", + "grok-mermaid": "0.2.2", "highlight.js": "10.7.3", "hosted-git-info": "9.0.3", "ignore": "7.0.5", @@ -561,8 +565,8 @@ "minimatch": "10.2.5", "proper-lockfile": "4.1.2", "semver": "7.8.0", - "typebox": "1.1.38", - "undici": "8.5.0", + "typebox": "1.3.7", + "undici": "8.9.0", "yaml": "2.9.0" }, "bin": { @@ -1038,14 +1042,16 @@ } }, "node_modules/@earendil-works/pi-coding-agent/node_modules/@earendil-works/pi-agent-core": { - "version": "0.80.9", - "resolved": "https://registry.npmjs.org/@earendil-works/pi-agent-core/-/pi-agent-core-0.80.9.tgz", + "version": "0.84.1", + "resolved": "https://registry.npmjs.org/@earendil-works/pi-agent-core/-/pi-agent-core-0.84.1.tgz", "dev": true, "license": "MIT", "dependencies": { - "@earendil-works/pi-ai": "^0.80.9", + "@earendil-works/pi-ai": "^0.84.1", + "@earendil-works/pi-telemetry": "^0.84.1", + "diff": "8.0.4", "ignore": "7.0.5", - "typebox": "1.1.38", + "typebox": "1.3.7", "yaml": "2.9.0" }, "engines": { @@ -1053,13 +1059,14 @@ } }, "node_modules/@earendil-works/pi-coding-agent/node_modules/@earendil-works/pi-ai": { - "version": "0.80.9", - "resolved": "https://registry.npmjs.org/@earendil-works/pi-ai/-/pi-ai-0.80.9.tgz", + "version": "0.84.1", + "resolved": "https://registry.npmjs.org/@earendil-works/pi-ai/-/pi-ai-0.84.1.tgz", "dev": true, "license": "MIT", "dependencies": { "@anthropic-ai/sdk": "0.91.1", "@aws-sdk/client-bedrock-runtime": "3.1048.0", + "@earendil-works/pi-telemetry": "^0.84.1", "@google/genai": "1.52.0", "@mistralai/mistralai": "2.2.6", "@opentelemetry/api": "1.9.0", @@ -1068,7 +1075,7 @@ "https-proxy-agent": "7.0.6", "openai": "6.26.0", "partial-json": "0.1.7", - "typebox": "1.1.38" + "typebox": "1.3.7" }, "bin": { "pi-ai": "dist/cli.js" @@ -1077,9 +1084,42 @@ "node": ">=22.19.0" } }, + "node_modules/@earendil-works/pi-coding-agent/node_modules/@earendil-works/pi-client": { + "version": "0.84.1", + "resolved": "https://registry.npmjs.org/@earendil-works/pi-client/-/pi-client-0.84.1.tgz", + "dev": true, + "license": "MIT", + "dependencies": { + "@earendil-works/pi-protocol": "^0.84.1" + }, + "engines": { + "node": ">=22.19.0" + } + }, + "node_modules/@earendil-works/pi-coding-agent/node_modules/@earendil-works/pi-protocol": { + "version": "0.84.1", + "resolved": "https://registry.npmjs.org/@earendil-works/pi-protocol/-/pi-protocol-0.84.1.tgz", + "dev": true, + "license": "MIT", + "dependencies": { + "typebox": "1.3.7" + }, + "engines": { + "node": ">=22.19.0" + } + }, + "node_modules/@earendil-works/pi-coding-agent/node_modules/@earendil-works/pi-telemetry": { + "version": "0.84.1", + "resolved": "https://registry.npmjs.org/@earendil-works/pi-telemetry/-/pi-telemetry-0.84.1.tgz", + "dev": true, + "license": "MIT", + "engines": { + "node": ">=22.19.0" + } + }, "node_modules/@earendil-works/pi-coding-agent/node_modules/@earendil-works/pi-tui": { - "version": "0.80.9", - "resolved": "https://registry.npmjs.org/@earendil-works/pi-tui/-/pi-tui-0.80.9.tgz", + "version": "0.84.1", + "resolved": "https://registry.npmjs.org/@earendil-works/pi-tui/-/pi-tui-0.84.1.tgz", "dev": true, "license": "MIT", "dependencies": { @@ -1194,9 +1234,6 @@ "arm64" ], "dev": true, - "libc": [ - "glibc" - ], "license": "MIT", "optional": true, "os": [ @@ -1214,9 +1251,6 @@ "arm64" ], "dev": true, - "libc": [ - "musl" - ], "license": "MIT", "optional": true, "os": [ @@ -1234,9 +1268,6 @@ "riscv64" ], "dev": true, - "libc": [ - "glibc" - ], "license": "MIT", "optional": true, "os": [ @@ -1254,9 +1285,6 @@ "x64" ], "dev": true, - "libc": [ - "glibc" - ], "license": "MIT", "optional": true, "os": [ @@ -1274,9 +1302,6 @@ "x64" ], "dev": true, - "libc": [ - "musl" - ], "license": "MIT", "optional": true, "os": [ @@ -1645,16 +1670,16 @@ "license": "MIT" }, "node_modules/@earendil-works/pi-coding-agent/node_modules/brace-expansion": { - "version": "5.0.6", - "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-5.0.6.tgz", - "integrity": "sha512-kLpxurY4Z4r9sgMsyG0Z9uzsBlgiU/EFKhj/h91/8yHu0edo7XuixOIH3VcJ8kkxs6/jPzoI6U9Vj3WqbMQ94g==", + "version": "5.0.9", + "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-5.0.9.tgz", + "integrity": "sha512-ScQ4IuvIEF1TMlP7Zt+vjJ//9zlPb2SDcxWxM3bk8s6t6GGdJ7KO1dCcTidOPJKePW30LE/2cT7wCyPho9/Wxg==", "dev": true, "license": "MIT", "dependencies": { "balanced-match": "^4.0.2" }, "engines": { - "node": "18 || 20 || >=22" + "node": "20 || >=22" } }, "node_modules/@earendil-works/pi-coding-agent/node_modules/buffer-equal-constant-time": { @@ -1919,6 +1944,16 @@ "dev": true, "license": "ISC" }, + "node_modules/@earendil-works/pi-coding-agent/node_modules/grok-mermaid": { + "version": "0.2.2", + "resolved": "https://registry.npmjs.org/grok-mermaid/-/grok-mermaid-0.2.2.tgz", + "integrity": "sha512-XcJEP5dDC8liHBh52mlLjU18fNvu1ckFsu0QpIG3+APZ270fsj9wxpiA6cOURmbUEuoMVgjbC2+UYgTdCqqgzA==", + "dev": true, + "license": "Apache-2.0", + "engines": { + "node": ">=18" + } + }, "node_modules/@earendil-works/pi-coding-agent/node_modules/highlight.js": { "version": "10.7.3", "resolved": "https://registry.npmjs.org/highlight.js/-/highlight.js-10.7.3.tgz", @@ -2263,9 +2298,9 @@ } }, "node_modules/@earendil-works/pi-coding-agent/node_modules/protobufjs": { - "version": "7.6.4", - "resolved": "https://registry.npmjs.org/protobufjs/-/protobufjs-7.6.4.tgz", - "integrity": "sha512-RJJPTTpvFfHcWLkIa2JFWK4XvtSzS0yEWDmunqHXli1h3JlkbcQZXDZdcWxv+JK3Xsl5/UFDPZ0iGm7DAengYw==", + "version": "7.6.5", + "resolved": "https://registry.npmjs.org/protobufjs/-/protobufjs-7.6.5.tgz", + "integrity": "sha512-/FPD0nUc9jH6rfFjji9IBqOz4pcSE3CsT1m7Ep6Mdb0LxSUMj8hgl6GomOvZzpNpAqqGaXA0P3VSrZLFzIhQrw==", "dev": true, "hasInstallScript": true, "license": "BSD-3-Clause", @@ -2388,16 +2423,16 @@ "license": "0BSD" }, "node_modules/@earendil-works/pi-coding-agent/node_modules/typebox": { - "version": "1.1.38", - "resolved": "https://registry.npmjs.org/typebox/-/typebox-1.1.38.tgz", - "integrity": "sha512-pZ0aQPmMmXoUvSbeuWf/Hzsc+avNw/Zd6VeE8CFgkVGWyuHPJvqeJJDeJqLve+K70LvjYIoleGcoJHPT17cWoA==", + "version": "1.3.7", + "resolved": "https://registry.npmjs.org/typebox/-/typebox-1.3.7.tgz", + "integrity": "sha512-meKuifc33Pccx0O6PdIzYMq3Og8zvP4TIi/a+Bw3AEMZMxOD0+RHGQvpglEe6Zdy3wZ8nqn/j95h8LUZLk/6Hg==", "dev": true, "license": "MIT" }, "node_modules/@earendil-works/pi-coding-agent/node_modules/undici": { - "version": "8.5.0", - "resolved": "https://registry.npmjs.org/undici/-/undici-8.5.0.tgz", - "integrity": "sha512-xamtWoB1EshgjpmlXd7GGm2VfdDtw1+rD8uhry8pSNW3If6S8E0m2T2+orSKeZXEn/aPJMviCpDBA65WJt8zhg==", + "version": "8.9.0", + "resolved": "https://registry.npmjs.org/undici/-/undici-8.9.0.tgz", + "integrity": "sha512-aWZpUj7XoGonMClx4gdDRfgBjqeA+F473aDmROQQbM9n6PRfK/u1q/a0X4wMTgcHfT8H6fpbt98PFuDUwFg2YA==", "dev": true, "license": "MIT", "engines": { @@ -2511,10 +2546,20 @@ "zod": "^3.25.28 || ^4" } }, + "node_modules/@earendil-works/pi-telemetry": { + "version": "0.84.1", + "resolved": "https://registry.npmjs.org/@earendil-works/pi-telemetry/-/pi-telemetry-0.84.1.tgz", + "integrity": "sha512-180/xGJtsq7IoR3p9EKWjRd0e9M4DkxInhlo9xyD7prDC7Qrhqq+nhvwrW0lFjPfXcEI2FSHmGCSyvSJE9GsaQ==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">=22.19.0" + } + }, "node_modules/@earendil-works/pi-tui": { - "version": "0.80.9", - "resolved": "https://registry.npmjs.org/@earendil-works/pi-tui/-/pi-tui-0.80.9.tgz", - "integrity": "sha512-unPTW8hRgIHEGjV8mJJ2jqm+fzgnRubes6V2FPk9ay1W9ZLofcpYQ3NDfrODXSci+oKbBpX9JyYUMfQV6jCA/A==", + "version": "0.84.1", + "resolved": "https://registry.npmjs.org/@earendil-works/pi-tui/-/pi-tui-0.84.1.tgz", + "integrity": "sha512-udeXFbgEhJ6JiB0uguwNVNkDy2FENfmtQwPcY+/iJ8GWeq18wkal1tKqa5YyeH0IqtX1vG0cGh8zfSYzyzVuLA==", "dev": true, "license": "MIT", "dependencies": { @@ -3783,9 +3828,9 @@ } }, "node_modules/typebox": { - "version": "1.1.38", - "resolved": "https://registry.npmjs.org/typebox/-/typebox-1.1.38.tgz", - "integrity": "sha512-pZ0aQPmMmXoUvSbeuWf/Hzsc+avNw/Zd6VeE8CFgkVGWyuHPJvqeJJDeJqLve+K70LvjYIoleGcoJHPT17cWoA==", + "version": "1.3.7", + "resolved": "https://registry.npmjs.org/typebox/-/typebox-1.3.7.tgz", + "integrity": "sha512-meKuifc33Pccx0O6PdIzYMq3Og8zvP4TIi/a+Bw3AEMZMxOD0+RHGQvpglEe6Zdy3wZ8nqn/j95h8LUZLk/6Hg==", "dev": true, "license": "MIT" }, diff --git a/package.json b/package.json index 02b19bf..80c3512 100644 --- a/package.json +++ b/package.json @@ -27,6 +27,7 @@ "README.md", "LICENSE", "CHANGELOG.md", + "docs/validation.md", "assets/pi-queue-steer-demo.gif" ], "scripts": { @@ -45,9 +46,9 @@ "@earendil-works/pi-tui": "*" }, "devDependencies": { - "@earendil-works/pi-ai": "0.80.9", - "@earendil-works/pi-coding-agent": "0.80.9", - "@earendil-works/pi-tui": "0.80.9", + "@earendil-works/pi-ai": "*", + "@earendil-works/pi-coding-agent": "*", + "@earendil-works/pi-tui": "*", "@types/node": "24.12.4", "tsx": "4.22.1", "typescript": "5.9.3" diff --git a/queue-state.ts b/queue-state.ts index 1db74f3..8c95023 100644 --- a/queue-state.ts +++ b/queue-state.ts @@ -175,6 +175,25 @@ export class DeliveryQueue { return this.items.length; } + /** Restore an in-memory queue snapshot without changing row identity or recency. */ + restore(items: readonly QueuedMessage[]): void { + const ids = new Set(); + let highestIdNumber = 0; + let highestSequence = 0; + const restored: QueuedMessage[] = []; + for (const item of items) { + if (ids.has(item.id)) throw new Error(`Duplicate queued row ID: ${item.id}`); + ids.add(item.id); + const idNumber = /-(\d+)$/.exec(item.id)?.[1]; + if (idNumber) highestIdNumber = Math.max(highestIdNumber, Number.parseInt(idNumber, 10)); + highestSequence = Math.max(highestSequence, item.sequence); + restored.push(this.copy(item)); + } + this.items = restored; + this.nextIdNumber = highestIdNumber + 1; + this.nextSequence = highestSequence + 1; + } + clear(): void { this.items = []; } diff --git a/queued-input.ts b/queued-input.ts index 0e766ec..41e1d0a 100644 --- a/queued-input.ts +++ b/queued-input.ts @@ -10,7 +10,7 @@ import { const PI_BUILTIN_COMMANDS = new Set([ "settings", "model", "scoped-models", "export", "import", "share", "copy", "name", "session", "changelog", "hotkeys", "fork", "clone", "tree", "trust", "login", "logout", "new", "compact", - "resume", "reload", "quit", + "resume", "reload", "quit", "debug", "arminsayshi", "dementedelves", ]); // Pi does not export its prompt argument parser or substitution helper. @@ -40,10 +40,13 @@ function parseCommandArgs(argsString: string): string[] { function substituteArgs(content: string, args: readonly string[]): string { const allArgs = args.join(" "); return content.replace( - /\$\{(\d+):-([^}]*)\}|\$\{@:(\d+)(?::(\d+))?\}|\$(ARGUMENTS|@|\d+)/g, + /\$\{(\d+|ARGUMENTS|@):-([^}]*)\}|\$\{@:(\d+)(?::(\d+))?\}|\$(ARGUMENTS|@|\d+)/g, (_match, defaultTarget, defaultValue, sliceStart, sliceLength, simple: string | undefined) => { if (defaultTarget) { - return args[Number.parseInt(defaultTarget, 10) - 1] || defaultValue; + const value = defaultTarget === "@" || defaultTarget === "ARGUMENTS" + ? allArgs + : args[Number.parseInt(defaultTarget, 10) - 1]; + return value || defaultValue; } if (sliceStart) { const start = Math.max(0, Number.parseInt(sliceStart, 10) - 1); @@ -58,6 +61,28 @@ function substituteArgs(content: string, args: readonly string[]): string { ); } +/** Whether Pi's TUI parks this submit in its private post-compaction queue. */ +export function queuesDuringCompaction( + text: string, + commands: readonly SlashCommandInfo[], + behavior: "submit" | "followUp" = "submit", +): boolean { + const normalized = text.trim(); + if (!normalized) return false; + const invocation = /^\/([^\s]+)/.exec(normalized); + const name = invocation?.[1]; + const extensionCommand = name + ? commands.some((command) => command.source === "extension" && command.name === name) + : false; + // Pi's follow-up action parks everything except extension commands. Regular + // submit executes bash, built-ins and extension commands before its queue. + if (behavior === "followUp") return !extensionCommand; + if (normalized.startsWith("!")) return false; + if (!name) return true; + if (PI_BUILTIN_COMMANDS.has(name)) return false; + return !extensionCommand; +} + export function expandQueuedInput(text: string, commands: readonly SlashCommandInfo[]): string { const invocation = text.match(/^\/([^\s]+)(?:\s+([\s\S]*))?$/); const name = invocation?.[1]; diff --git a/test/fixtures/prompts/review.md b/test/fixtures/prompts/review.md new file mode 100644 index 0000000..14f6a08 --- /dev/null +++ b/test/fixtures/prompts/review.md @@ -0,0 +1,4 @@ +--- +description: Queue evidence prompt +--- +PROMPT EXPANDED: first=$1 all=$ARGUMENTS default=${3:-fallback} diff --git a/test/fixtures/skills/bro/SKILL.md b/test/fixtures/skills/bro/SKILL.md new file mode 100644 index 0000000..5880080 --- /dev/null +++ b/test/fixtures/skills/bro/SKILL.md @@ -0,0 +1,5 @@ +--- +name: bro +description: Queue evidence skill +--- +SKILL EXPANDED BODY diff --git a/test/fixtures/tui-faux-provider.ts b/test/fixtures/tui-faux-provider.ts new file mode 100644 index 0000000..592e77c --- /dev/null +++ b/test/fixtures/tui-faux-provider.ts @@ -0,0 +1,94 @@ +import { appendFileSync, existsSync, mkdirSync } from "node:fs"; +import { join } from "node:path"; +import { + fauxAssistantMessage, + fauxProvider, + type FauxResponseFactory, +} from "@earendil-works/pi-ai/compat"; +import type { ExtensionAPI } from "@earendil-works/pi-coding-agent"; + +const stateDir = process.env.PI_QUEUE_TUI_STATE_DIR ?? "/tmp/pi-queue-steer-tui"; +const configuredContextWindow = Number.parseInt(process.env.PI_QUEUE_TUI_CONTEXT_WINDOW ?? "100000", 10); +const contextWindow = Number.isFinite(configuredContextWindow) && configuredContextWindow > 0 + ? configuredContextWindow + : 100_000; +const pathInState = (name: string): string => join(stateDir, name); +const faux = fauxProvider({ + tokensPerSecond: 10_000, + models: [{ id: "queue-e2e", contextWindow, maxTokens: 100 }], +}); + +function contentText(content: Parameters[0] | unknown): string { + if (typeof content === "string") return content; + if (!Array.isArray(content)) return ""; + return content + .filter((part) => typeof part === "object" && part !== null && "type" in part && part.type === "text" && "text" in part) + .map((part) => typeof part === "object" && part !== null && "text" in part && typeof part.text === "string" ? part.text : "") + .join("\n"); +} + +const respond: FauxResponseFactory = async (context, options, state) => { + const userTexts: string[] = []; + for (const message of context.messages) { + if (message.role === "user") userTexts.push(contentText(message.content)); + } + const lastUser = userTexts.at(-1) ?? ""; + appendFileSync( + pathInState("provider-calls.jsonl"), + `${JSON.stringify({ + call: state.callCount, + length: lastUser.length, + prefix: lastUser.slice(0, 160), + userPrefixes: userTexts.map((text) => text.slice(0, 160)), + })}\n`, + ); + + if (lastUser.includes("conversation to summarize") || lastUser.includes("NEW conversation messages")) { + while (existsSync(pathInState("hold-summary")) && !existsSync(pathInState("release-summary"))) { + if (options?.signal?.aborted) return fauxAssistantMessage("", { stopReason: "aborted" }); + await new Promise((resolve) => setTimeout(resolve, 20)); + } + if (existsSync(pathInState("fail-summary"))) throw new Error("synthetic TUI summary failure"); + return fauxAssistantMessage("FAUX COMPACTION SUMMARY"); + } + + const gate = /^BLOCK:([a-z0-9-]+)/.exec(lastUser)?.[1]; + if (gate) { + while (!existsSync(pathInState(`gate-${gate}`))) { + if (options?.signal?.aborted) return fauxAssistantMessage("", { stopReason: "aborted" }); + await new Promise((resolve) => setTimeout(resolve, 20)); + } + } + return fauxAssistantMessage(`FAUX RESPONSE: ${lastUser.slice(0, 160)}`); +}; + +faux.setResponses(Array.from({ length: 200 }, () => respond)); + +export default function tuiFauxProvider(pi: ExtensionAPI): void { + mkdirSync(stateDir, { recursive: true }); + appendFileSync(pathInState("runtime-inits.log"), `${Date.now()}\n`); + pi.on("session_before_compact", (event) => { + appendFileSync( + pathInState("events.jsonl"), + `${JSON.stringify({ event: "session_before_compact", reason: event.reason })}\n`, + ); + }); + const model = faux.getModel(); + pi.registerProvider(model.provider, { + name: "Queue evidence faux provider", + baseUrl: model.baseUrl, + apiKey: "queue-evidence-key", + api: model.api, + streamSimple: faux.provider.streamSimple, + models: [{ + id: model.id, + name: model.name, + api: model.api, + reasoning: model.reasoning, + input: model.input, + cost: model.cost, + contextWindow: model.contextWindow, + maxTokens: model.maxTokens, + }], + }); +} diff --git a/test/pi-integration.test.ts b/test/pi-integration.test.ts new file mode 100644 index 0000000..5073f92 --- /dev/null +++ b/test/pi-integration.test.ts @@ -0,0 +1,376 @@ +import assert from "node:assert/strict"; +import { mkdtempSync, mkdirSync, rmSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import test from "node:test"; +import { + fauxAssistantMessage, + fauxProvider, + type FauxProviderHandle, +} from "@earendil-works/pi-ai/compat"; +import { + type AgentSession, + type AgentSessionEvent, + createAgentSession, + DefaultResourceLoader, + SessionManager, + SettingsManager, + type ExtensionFactory, +} from "@earendil-works/pi-coding-agent"; +import queueSteerExtension from "../index.ts"; + +type CompactionEndEvent = Extract; +type AgentStartEvent = Extract; + +interface IntegrationHarness { + session: AgentSession; + faux: FauxProviderHandle; + cleanup(): Promise; +} + +function nextCompactionEnd(session: AgentSession): Promise { + return new Promise((resolve) => { + let unsubscribe: (() => void) | undefined; + unsubscribe = session.subscribe((event) => { + if (event.type !== "compaction_end") return; + unsubscribe?.(); + resolve(event); + }); + }); +} + +function nextAgentStart(session: AgentSession): Promise { + return new Promise((resolve) => { + let unsubscribe: (() => void) | undefined; + unsubscribe = session.subscribe((event) => { + if (event.type !== "agent_start") return; + unsubscribe?.(); + resolve(event); + }); + }); +} + +function nextAgentRun(session: AgentSession): Promise { + return new Promise((resolve) => { + let started = false; + let unsubscribe: (() => void) | undefined; + unsubscribe = session.subscribe((event) => { + if (event.type === "agent_start") { + started = true; + return; + } + if (event.type !== "agent_settled" || !started) return; + unsubscribe?.(); + resolve(); + }); + }); +} + +function nextAgentRunForUser(session: AgentSession, expected: string): Promise { + return new Promise((resolve) => { + let matched = false; + let unsubscribe: (() => void) | undefined; + unsubscribe = session.subscribe((event) => { + if (event.type === "message_start" && event.message.role === "user") { + const text = typeof event.message.content === "string" + ? event.message.content + : event.message.content + .filter((part) => part.type === "text") + .map((part) => part.text) + .join("\n"); + if (text === expected) matched = true; + return; + } + if (event.type !== "agent_settled" || !matched) return; + unsubscribe?.(); + resolve(); + }); + }); +} + +async function within(promise: Promise, detail: () => string): Promise { + let timer: ReturnType | undefined; + try { + return await Promise.race([ + promise, + new Promise((_resolve, reject) => { + timer = setTimeout(() => reject(new Error(`Timed out: ${detail()}`)), 2_000); + }), + ]); + } finally { + if (timer) clearTimeout(timer); + } +} + +function userTexts(session: AgentSession): string[] { + return session.messages + .filter((message) => message.role === "user") + .map((message) => { + if (typeof message.content === "string") return message.content; + return message.content + .filter((part) => part.type === "text") + .map((part) => part.text) + .join("\n"); + }); +} + +async function createIntegrationHarness(options: { + contextWindow?: number; + maxTokens?: number; + extraExtensions?: ExtensionFactory[]; + retryEnabled?: boolean; +} = {}): Promise { + const cwd = mkdtempSync(join(tmpdir(), "pi-queue-integration-")); + const agentDir = join(cwd, "agent"); + mkdirSync(agentDir, { recursive: true }); + const faux = fauxProvider({ + models: [{ + id: "queue-integration", + contextWindow: options.contextWindow ?? 100_000, + maxTokens: options.maxTokens ?? 1_000, + }], + }); + const model = faux.getModel(); + const settingsManager = SettingsManager.inMemory({ + compaction: { enabled: true, keepRecentTokens: 1, reserveTokens: 0 }, + retry: options.retryEnabled + ? { enabled: true, maxRetries: 2, baseDelayMs: 1 } + : { enabled: false }, + }); + const sessionManager = SessionManager.inMemory(cwd); + const providerExtension: ExtensionFactory = (pi) => { + pi.registerProvider(model.provider, { + name: "Faux integration provider", + baseUrl: model.baseUrl, + apiKey: "integration-test-key", + api: model.api, + streamSimple: faux.provider.streamSimple, + models: [{ + id: model.id, + name: model.name, + api: model.api, + reasoning: model.reasoning, + input: model.input, + cost: model.cost, + contextWindow: model.contextWindow, + maxTokens: model.maxTokens, + }], + }); + }; + const resourceLoader = new DefaultResourceLoader({ + cwd, + agentDir, + settingsManager, + extensionFactories: [providerExtension, queueSteerExtension, ...(options.extraExtensions ?? [])], + noSkills: true, + noPromptTemplates: true, + noThemes: true, + noContextFiles: true, + }); + await resourceLoader.reload(); + const { session } = await createAgentSession({ + cwd, + agentDir, + model, + settingsManager, + sessionManager, + resourceLoader, + noTools: "all", + }); + await session.bindExtensions({ mode: "tui" }); + return { + session, + faux, + async cleanup() { + // Let the extension's public-API editor recomposition timer settle + // before invalidating its session context. + await new Promise((resolve) => setTimeout(resolve, 5)); + session.dispose(); + rmSync(cwd, { recursive: true, force: true }); + }, + }; +} + +function gatedResponse( + content: string, + options?: Parameters[1], +): { + step: () => Promise>; + release(): void; +} { + let releaseGate: (() => void) | undefined; + const gate = new Promise((resolve) => { + releaseGate = resolve; + }); + return { + step: async () => { + await gate; + return fauxAssistantMessage(content, options); + }, + release() { + releaseGate?.(); + }, + }; +} + +async function seedSession(harness: IntegrationHarness): Promise { + harness.faux.setResponses([ + fauxAssistantMessage("seed response one"), + fauxAssistantMessage("seed response two"), + ]); + await harness.session.prompt("seed one"); + await harness.session.prompt("seed two"); +} + +test("real AgentSession runs a queued manual compaction before the following row", async () => { + const harness = await createIntegrationHarness(); + try { + await seedSession(harness); + const active = gatedResponse("active response"); + harness.faux.setResponses([ + active.step, + fauxAssistantMessage("manual summary"), + fauxAssistantMessage("manual split-turn summary"), + fauxAssistantMessage("response after compaction"), + ]); + const activeStarted = nextAgentStart(harness.session); + const activePrompt = harness.session.prompt("active prompt"); + await within(activeStarted, () => "manual compaction agent did not start"); + await harness.session.prompt("/compact preserve integration evidence", { streamingBehavior: "followUp" }); + await harness.session.prompt("after manual compaction", { streamingBehavior: "followUp" }); + const compactionEnded = nextCompactionEnd(harness.session); + const resumed = nextAgentRun(harness.session); + active.release(); + await activePrompt; + + const compaction = await within(compactionEnded, () => "manual compaction did not finish"); + assert.equal(compaction.reason, "manual"); + assert.equal(compaction.result?.summary.includes("manual summary"), true); + await within(resumed, () => "post-compaction row did not run"); + assert.equal(userTexts(harness.session).at(-1), "after manual compaction"); + assert.equal(userTexts(harness.session).filter((text) => text === "after manual compaction").length, 1); + assert.equal(harness.session.getLastAssistantText(), "response after compaction"); + assert.equal(harness.session.sessionManager.getEntries().filter((entry) => entry.type === "compaction").length, 1); + } finally { + await harness.cleanup(); + } +}); + +test("real failed manual compaction releases the following row without adding a compaction entry", async () => { + const harness = await createIntegrationHarness(); + try { + await seedSession(harness); + const active = gatedResponse("active response"); + harness.faux.setResponses([ + active.step, + () => { + throw new Error("synthetic summary failure"); + }, + fauxAssistantMessage("response after failed compaction"), + ]); + const activeStarted = nextAgentStart(harness.session); + const activePrompt = harness.session.prompt("active before failure"); + await within(activeStarted, () => "failed-compaction agent did not start"); + await harness.session.prompt("/compact", { streamingBehavior: "followUp" }); + await harness.session.prompt("after failed compaction", { streamingBehavior: "followUp" }); + const compactionEnded = nextCompactionEnd(harness.session); + const resumed = nextAgentRun(harness.session); + active.release(); + await activePrompt; + + const compaction = await within(compactionEnded, () => "failed compaction did not finish"); + assert.equal(compaction.reason, "manual"); + assert.match(compaction.errorMessage ?? "", /synthetic summary failure/); + await within(resumed, () => "row after failed compaction did not run"); + assert.equal(userTexts(harness.session).filter((text) => text === "after failed compaction").length, 1); + assert.equal(harness.session.getLastAssistantText(), "response after failed compaction"); + assert.equal(harness.session.sessionManager.getEntries().some((entry) => entry.type === "compaction"), false); + } finally { + await harness.cleanup(); + } +}); + +test("real retry finishes before the extension releases its queued follow-up", async () => { + const harness = await createIntegrationHarness({ retryEnabled: true }); + try { + const trace: string[] = []; + harness.session.subscribe((event) => trace.push(event.type)); + const failed = gatedResponse("", { + stopReason: "error", + errorMessage: "rate limit exceeded", + }); + harness.faux.setResponses([ + failed.step, + fauxAssistantMessage("retry succeeded"), + fauxAssistantMessage("queued follow-up succeeded"), + ]); + const started = nextAgentStart(harness.session); + const prompt = harness.session.prompt("retry original"); + await within(started, () => trace.join(", ")); + await harness.session.prompt("after retry", { streamingBehavior: "followUp" }); + failed.release(); + await within(prompt, () => trace.join(", ")); + + assert.ok(trace.includes("auto_retry_start")); + assert.ok(trace.includes("auto_retry_end")); + assert.equal(userTexts(harness.session).filter((text) => text === "after retry").length, 1); + assert.equal(harness.session.getLastAssistantText(), "queued follow-up succeeded"); + } finally { + await harness.cleanup(); + } +}); + +test("real public prompt path triggers overflow compaction and preserves a queued follow-up", async () => { + const summaryExtension: ExtensionFactory = (pi) => { + pi.on("session_before_compact", (event) => { + if (event.reason !== "overflow") return; + return { + compaction: { + summary: "overflow integration summary", + firstKeptEntryId: event.preparation.firstKeptEntryId, + tokensBefore: event.preparation.tokensBefore, + details: { source: "integration-test" }, + }, + }; + }); + }; + const harness = await createIntegrationHarness({ + contextWindow: 1_000, + maxTokens: 100, + extraExtensions: [summaryExtension], + }); + try { + const trace: string[] = []; + harness.session.subscribe((event) => { + if (event.type === "message_end" && event.message.role === "assistant") { + trace.push(`${event.type}:${event.message.stopReason}:${event.message.errorMessage ?? ""}`); + return; + } + trace.push(event.type); + }); + const active = gatedResponse("partial response"); + harness.faux.setResponses([ + active.step, + fauxAssistantMessage("completed queued follow-up"), + ]); + const activeStarted = nextAgentStart(harness.session); + const compactionEnded = nextCompactionEnd(harness.session); + const prompt = harness.session.prompt("x".repeat(20_000)); + await within(activeStarted, () => trace.join(", ")); + await harness.session.prompt("queued across overflow", { streamingBehavior: "followUp" }); + const queuedRun = nextAgentRunForUser(harness.session, "queued across overflow"); + active.release(); + await within(prompt, () => trace.join(", ")); + + const compaction = await within(compactionEnded, () => trace.join(", ")); + await within(queuedRun, () => trace.join(", ")); + assert.equal(compaction.reason, "overflow"); + assert.equal(compaction.willRetry, false); + assert.equal(compaction.result?.summary, "overflow integration summary"); + assert.equal(userTexts(harness.session).filter((text) => text === "queued across overflow").length, 1); + assert.equal(harness.faux.state.callCount, 2); + assert.equal(harness.session.getLastAssistantText(), "completed queued follow-up"); + } finally { + await harness.cleanup(); + } +}); diff --git a/test/queue-state.test.ts b/test/queue-state.test.ts index 6adb468..e129fdb 100644 --- a/test/queue-state.test.ts +++ b/test/queue-state.test.ts @@ -3,7 +3,8 @@ import { mkdtempSync, mkdirSync, rmSync, writeFileSync } from "node:fs"; import { tmpdir } from "node:os"; import { join } from "node:path"; import test from "node:test"; -import type { SlashCommandInfo } from "@earendil-works/pi-coding-agent"; +import type { ImageContent } from "@earendil-works/pi-ai"; +import type { CompactOptions, SlashCommandInfo } from "@earendil-works/pi-coding-agent"; import { visibleWidth } from "@earendil-works/pi-tui"; import queueSteerExtension from "../index.ts"; import { DeliveryQueue, QueueEditSession, type QueueLane } from "../queue-state.ts"; @@ -61,6 +62,28 @@ test("restores failed batches at the front in their original order", () => { assert.deepEqual(queue.laneSnapshot("followUp").map((item) => item.text), ["first", "second", "third"]); }); +test("restores queue snapshots with stable IDs, recency, images, and collision-free counters", () => { + const original = new DeliveryQueue(); + const first = original.enqueue("steer", "first", ["one.png"]); + const mostRecent = original.enqueue("followUp", "second", ["two.png"]); + const snapshot = original.snapshot(); + + const restored = new DeliveryQueue(); + restored.restore(snapshot); + assert.deepEqual(restored.snapshot(), snapshot); + assert.equal(restored.mostRecentId(), mostRecent.id); + assert.equal(restored.get(first.id)?.images[0], "one.png"); + const next = restored.enqueue("steer", "third"); + assert.equal(next.id, "steer-3"); + assert.ok(next.sequence > mostRecent.sequence); +}); + +test("rejects duplicate row IDs in restored snapshots", () => { + const queue = new DeliveryQueue(); + const row = queue.enqueue("steer", "one"); + assert.throws(() => queue.restore([row, { ...row, text: "duplicate" }]), /Duplicate queued row ID/); +}); + test("edit sessions keep cross-lane drafts private until commit", () => { const queue = new DeliveryQueue(); const steer = queue.enqueue("steer", "steer original"); @@ -135,9 +158,18 @@ test("toggling a lane twice leaves the row untouched at commit", () => { class MockEditor { private text = ""; + private autocompleteVisible = false; onSubmit?: (text: string) => void; onChange?: (text: string) => void; + setAutocompleteVisible(visible: boolean): void { + this.autocompleteVisible = visible; + } + + isShowingAutocomplete(): boolean { + return this.autocompleteVisible; + } + getText(): string { return this.text; } @@ -157,20 +189,42 @@ class MockEditor { invalidate(): void {} } +const DEFAULT_TEST_CWD = mkdtempSync(join(tmpdir(), "pi-queue-steer-default-")); +mkdirSync(join(DEFAULT_TEST_CWD, ".pi")); +writeFileSync( + join(DEFAULT_TEST_CWD, ".pi", "settings.json"), + JSON.stringify({ steeringMode: "one-at-a-time", followUpMode: "one-at-a-time" }), +); +test.after(() => rmSync(DEFAULT_TEST_CWD, { recursive: true, force: true })); + function createHarness(options: { cwd?: string; projectTrusted?: boolean; commands?: SlashCommandInfo[]; + mode?: "tui" | "rpc" | "json" | "print"; + sendFailureAt?: number; + compactStartError?: Error; + autocompleteVisible?: boolean; } = {}) { type Handler = (event: any, context: any) => any; const handlers = new Map(); const sent: Array<{ content: unknown; options: any }> = []; + const submitted: string[] = []; + const compactCalls: CompactOptions[] = []; const notifications: Array<{ message: string; level: string }> = []; let idle = false; let pending = false; let aborted = false; - let activeEditor = new MockEditor(); - let currentFactory: any = () => activeEditor; + const createDefaultEditor = (): MockEditor => { + const editor = new MockEditor(); + editor.setAutocompleteVisible(options.autocompleteVisible ?? false); + editor.onSubmit = (text) => submitted.push(text); + return editor; + }; + type MockEditorFactory = (_tui: unknown, _theme: unknown, _keybindings: unknown) => MockEditor; + let activeEditor = createDefaultEditor(); + let currentFactory: MockEditorFactory = () => createDefaultEditor(); + let editorInstallCount = 0; let widget: unknown; const keybindings = { @@ -186,7 +240,8 @@ function createHarness(options: { const ui = { getEditorComponent: () => currentFactory, - setEditorComponent(factory: any) { + setEditorComponent(factory: MockEditorFactory) { + editorInstallCount += 1; currentFactory = factory; activeEditor = factory({}, {}, keybindings); }, @@ -200,17 +255,22 @@ function createHarness(options: { }, }; + const mode = options.mode ?? "tui"; const context = { - mode: "tui", - hasUI: true, - cwd: options.cwd ?? "/tmp", + mode, + hasUI: mode === "tui" || mode === "rpc", + cwd: options.cwd ?? DEFAULT_TEST_CWD, ui, isIdle: () => idle, - isProjectTrusted: () => options.projectTrusted ?? false, + isProjectTrusted: () => options.projectTrusted ?? true, hasPendingMessages: () => pending, abort() { aborted = true; }, + compact(compactOptions: CompactOptions = {}) { + if (options.compactStartError) throw options.compactStartError; + compactCalls.push(compactOptions); + }, }; const pi = { @@ -219,9 +279,10 @@ function createHarness(options: { registered.push(handler); handlers.set(name, registered); }, - sendUserMessage(content: unknown, options?: unknown) { - sent.push({ content, options }); - if (options) pending = true; + sendUserMessage(content: unknown, sendOptions?: unknown) { + if (options.sendFailureAt === sent.length + 1) throw new Error("synthetic send failure"); + sent.push({ content, options: sendOptions }); + if (sendOptions) pending = true; }, getCommands: () => options.commands ?? [], }; @@ -230,8 +291,11 @@ function createHarness(options: { const emit = async (name: string, event: any = {}): Promise => { const results = []; + const emittedEvent = name === "agent_end" && event.messages === undefined + ? { ...event, messages: [] } + : event; for (const handler of handlers.get(name) ?? []) { - results.push(await handler(event, context)); + results.push(await handler(emittedEvent, context)); } return results; }; @@ -239,6 +303,8 @@ function createHarness(options: { return { emit, sent, + submitted, + compactCalls, notifications, get editor() { return activeEditor; @@ -246,6 +312,12 @@ function createHarness(options: { get widget() { return widget; }, + get editorInstallCount() { + return editorInstallCount; + }, + get editorFactory() { + return currentFactory; + }, get aborted() { return aborted; }, @@ -258,6 +330,12 @@ function createHarness(options: { replaceEditor(editor = new MockEditor()) { ui.setEditorComponent(() => editor); }, + wrapEditorFactory() { + const wrappedFactory = currentFactory; + ui.setEditorComponent((tui, theme, editorKeybindings) => ( + wrappedFactory(tui, theme, editorKeybindings) + )); + }, }; } @@ -279,6 +357,14 @@ function renderWidget(harness: ReturnType, width = 76): st return component.render(width).join("\n"); } +async function waitFor(predicate: () => boolean): Promise { + for (let attempt = 0; attempt < 20; attempt += 1) { + if (predicate()) return; + await new Promise((resolve) => setTimeout(resolve, 0)); + } + assert.fail("Timed out waiting for condition"); +} + test("renders stacked lane boxes with steering above follow-ups", async () => { const harness = createHarness(); await harness.emit("session_start"); @@ -369,6 +455,80 @@ test("injects follow-ups through Pi's native continuation queue at agent_end", a assert.match(renderWidget(harness), /later two/); }); +test("restores only the unsent tail after a synchronous all-mode batch failure", async () => { + const cwd = mkdtempSync(join(tmpdir(), "pi-queue-partial-send-")); + mkdirSync(join(cwd, ".pi")); + writeFileSync(join(cwd, ".pi", "settings.json"), JSON.stringify({ followUpMode: "all" })); + const harness = createHarness({ cwd, projectTrusted: true, sendFailureAt: 2 }); + try { + await harness.emit("session_start"); + await enqueue(harness, "followUp", "accepted first"); + await enqueue(harness, "followUp", "restore second"); + await enqueue(harness, "followUp", "restore third"); + + await harness.emit("agent_end"); + assert.deepEqual(harness.sent.map((item) => item.content), ["accepted first"]); + const rendered = renderWidget(harness); + assert.doesNotMatch(rendered, /accepted first/); + assert.match(rendered, /restore second/); + assert.match(rendered, /restore third/); + assert.match(harness.notifications.at(-1)?.message ?? "", /synthetic send failure/); + } finally { + rmSync(cwd, { recursive: true, force: true }); + } +}); + +test("restores an idle row after a synchronous send failure", async () => { + const harness = createHarness({ sendFailureAt: 1 }); + await harness.emit("session_start"); + harness.setIdle(true); + await enqueue(harness, "followUp", "retry me"); + + await harness.emit("agent_settled"); + assert.equal(harness.sent.length, 0); + assert.match(renderWidget(harness), /retry me/); + assert.match(harness.notifications.at(-1)?.message ?? "", /synthetic send failure/); +}); + +test("delivers image-bearing command text as a message without dropping attachments", async () => { + const harness = createHarness(); + const image: ImageContent = { type: "image", data: "AA==", mimeType: "image/png" }; + await harness.emit("session_start"); + await harness.emit("input", { + source: "interactive", + text: "/reload", + images: [image], + streamingBehavior: "followUp", + }); + + assert.doesNotMatch(renderWidget(harness), /command row/); + await harness.emit("agent_end"); + assert.deepEqual(harness.sent[0], { + content: [{ type: "text", text: "/reload" }, image], + options: { deliverAs: "followUp" }, + }); + assert.deepEqual(harness.submitted, []); +}); + +test("does not take ownership of interactive-source input outside TUI mode", async () => { + const modes: ("rpc" | "json" | "print")[] = ["rpc", "json", "print"]; + for (const mode of modes) { + const harness = createHarness({ mode }); + await harness.emit("session_start", { reason: "startup" }); + const results = await harness.emit("input", { + source: "interactive", + text: "/reload", + streamingBehavior: "followUp", + }); + assert.deepEqual(results, [{ action: "continue" }]); + assert.equal(harness.sent.length, 0); + await new Promise((resolve) => setTimeout(resolve, 0)); + assert.equal(harness.compactCalls.length, 0); + assert.equal(harness.editorInstallCount, 0); + assert.equal(harness.widget, undefined); + } +}); + test("honours Pi all-mode settings and pins the whole edited lane", async () => { const cwd = mkdtempSync(join(tmpdir(), "pi-queue-steer-all-")); mkdirSync(join(cwd, ".pi")); @@ -399,6 +559,33 @@ test("honours Pi all-mode settings and pins the whole edited lane", async () => } }); +test("restarts an all-mode lane in FIFO order after it stays pinned through settle", async () => { + const cwd = mkdtempSync(join(tmpdir(), "pi-queue-steer-all-restart-")); + mkdirSync(join(cwd, ".pi")); + writeFileSync(join(cwd, ".pi", "settings.json"), JSON.stringify({ followUpMode: "all" })); + try { + const harness = createHarness({ cwd, projectTrusted: true }); + await harness.emit("session_start"); + await enqueue(harness, "followUp", "restart one"); + await enqueue(harness, "followUp", "restart two"); + harness.editor.handleInput("alt-up"); + await harness.emit("agent_end"); + assert.equal(harness.sent.length, 0); + + harness.setIdle(true); + await harness.emit("agent_settled"); + harness.editor.handleInput("enter"); + assert.deepEqual(harness.sent, [{ content: "restart one", options: undefined }]); + await harness.emit("agent_end"); + assert.deepEqual(harness.sent, [ + { content: "restart one", options: undefined }, + { content: "restart two", options: { deliverAs: "followUp" } }, + ]); + } finally { + rmSync(cwd, { recursive: true, force: true }); + } +}); + test("Alt+Up enters at the most recently enqueued row across both lanes", async () => { const harness = createHarness(); await harness.emit("session_start"); @@ -663,7 +850,7 @@ test("expands queued prompt templates and short Agent Skill commands at delivery { name: "skill:bro", source: "skill", sourceInfo: sourceInfo(skillPath) }, ], }); - const image = { type: "image", source: { type: "base64", mediaType: "image/png", data: "AA==" } }; + const image: ImageContent = { type: "image", data: "AA==", mimeType: "image/png" }; try { await harness.emit("session_start"); await harness.emit("input", { @@ -723,3 +910,321 @@ test("an expansion failure restores and pauses an entire all-mode batch", async rmSync(cwd, { recursive: true, force: true }); } }); + +test("keeps reload runnable when compact aborts a preflight prompt", async () => { + const harness = createHarness(); + await harness.emit("session_start"); + harness.setIdle(true); + await harness.emit("input", { source: "interactive", text: "native prompt" }); + + harness.editor.onSubmit?.("/compact keep the prompt details"); + assert.equal(harness.compactCalls[0]?.customInstructions, "keep the prompt details"); + harness.editor.onSubmit?.("/reload"); + assert.match(renderWidget(harness), /\/reload/); + + await harness.emit("turn_end", { message: { role: "assistant", stopReason: "aborted" } }); + harness.compactCalls[0]?.onError?.(new Error("summary failed")); + await waitFor(() => harness.submitted.length === 1); + assert.deepEqual(harness.submitted, ["/reload"]); +}); + +test("owns busy manual compaction so its abort does not pause queued rows", async () => { + const harness = createHarness(); + await harness.emit("session_start"); + await enqueue(harness, "followUp", "continue after compact"); + + harness.editor.onSubmit?.("/compact preserve the queue"); + assert.equal(harness.compactCalls[0]?.customInstructions, "preserve the queue"); + await harness.emit("turn_end", { message: { role: "assistant", stopReason: "aborted" } }); + assert.doesNotMatch(renderWidget(harness), /paused/); + + harness.setIdle(true); + harness.compactCalls[0]?.onComplete?.({ + summary: "summary", + firstKeptEntryId: "entry-1", + tokensBefore: 100, + estimatedTokensAfter: 20, + }); + await waitFor(() => harness.sent.length === 1); + assert.equal(harness.sent[0]?.content, "continue after compact"); +}); + +test("restores and pauses a command row when compaction cannot start", async () => { + const harness = createHarness({ compactStartError: new Error("cannot start") }); + await harness.emit("session_start"); + harness.setIdle(true); + await enqueue(harness, "followUp", "/compact"); + await enqueue(harness, "followUp", "after compact"); + + await harness.emit("agent_settled"); + const rendered = renderWidget(harness); + assert.match(rendered, /\/compact/); + assert.match(rendered, /after compact/); + assert.match(rendered, /paused/); + assert.match(harness.notifications.at(-1)?.message ?? "", /Could not start compaction: cannot start/); + assert.equal(harness.sent.length, 0); +}); + +test("leaves ordinary compaction input native and waits for its full run", async () => { + const harness = createHarness(); + await harness.emit("session_start"); + harness.setIdle(true); + harness.editor.onSubmit?.("/compact"); + assert.equal(harness.compactCalls.length, 1); + + harness.editor.onSubmit?.("ordinary native message"); + assert.deepEqual(harness.submitted, ["ordinary native message"]); + + harness.editor.setText("/reload"); + harness.editor.handleInput("alt-enter"); + assert.match(renderWidget(harness), /\/reload/); + + // Real standalone compaction reports idle before Pi's unawaited TUI queue + // flush has reached agent_start. The reload must remain held through it. + harness.compactCalls[0]?.onComplete?.({ + summary: "summary", + firstKeptEntryId: "entry-1", + tokensBefore: 100, + estimatedTokensAfter: 20, + }); + await new Promise((resolve) => setTimeout(resolve, 0)); + assert.deepEqual(harness.submitted, ["ordinary native message"]); + + harness.setIdle(false); + await harness.emit("turn_start"); + harness.setIdle(true); + await harness.emit("agent_settled"); + await waitFor(() => harness.submitted.length === 2); + assert.deepEqual(harness.submitted, ["ordinary native message", "/reload"]); +}); + +test("holds a follow-up at a length stop so Pi can decide whether to auto-compact", async () => { + const harness = createHarness(); + await harness.emit("session_start"); + await enqueue(harness, "followUp", "after overflow"); + await harness.emit("agent_end", { + messages: [{ role: "assistant", stopReason: "length", content: [] }], + }); + assert.equal(harness.sent.length, 0); + + await harness.emit("session_before_compact", { reason: "overflow" }); + harness.setIdle(true); + await harness.emit("agent_settled"); + await waitFor(() => harness.sent.length === 1); + assert.equal(harness.sent[0]?.content, "after overflow"); +}); + +test("releases a length-stop hold at settle when Pi does not compact", async () => { + const harness = createHarness(); + await harness.emit("session_start"); + await enqueue(harness, "followUp", "after full-length output"); + await harness.emit("agent_end", { + messages: [{ role: "assistant", stopReason: "length", content: [] }], + }); + harness.setIdle(true); + await harness.emit("agent_settled"); + assert.equal(harness.sent[0]?.content, "after full-length output"); +}); + +test("holds reload through automatic compaction until the agent settles", async () => { + const harness = createHarness(); + await harness.emit("session_start"); + harness.setIdle(true); + await harness.emit("session_before_compact", { reason: "overflow" }); + harness.editor.onSubmit?.("/reload"); + assert.equal(harness.submitted.length, 0); + + await harness.emit("agent_settled"); + await waitFor(() => harness.submitted.length === 1); + assert.deepEqual(harness.submitted, ["/reload"]); +}); + +test("captures a compaction command while slash autocomplete is visible", async () => { + const harness = createHarness({ autocompleteVisible: true }); + await harness.emit("session_start"); + harness.setIdle(true); + await harness.emit("session_before_compact", { reason: "threshold" }); + harness.editor.setText("/reload"); + harness.editor.handleInput("alt-enter"); + + assert.match(renderWidget(harness), /\/reload/); + assert.deepEqual(harness.submitted, []); + await harness.emit("agent_settled"); + await waitFor(() => harness.submitted.length === 1); + assert.deepEqual(harness.submitted, ["/reload"]); +}); + +test("holds automatic-compaction commands through ordinary native input", async () => { + const harness = createHarness(); + await harness.emit("session_start"); + harness.setIdle(true); + await harness.emit("session_before_compact", { reason: "threshold" }); + harness.editor.onSubmit?.("ordinary after automatic compaction"); + harness.editor.onSubmit?.("/reload"); + + await harness.emit("agent_settled"); + await new Promise((resolve) => setTimeout(resolve, 0)); + assert.deepEqual(harness.submitted, ["ordinary after automatic compaction"]); + + harness.setIdle(false); + await harness.emit("turn_start"); + harness.setIdle(true); + await harness.emit("agent_settled"); + await waitFor(() => harness.submitted.length === 2); + assert.deepEqual(harness.submitted, ["ordinary after automatic compaction", "/reload"]); +}); + +test("snapshots rows arriving after reload scheduling at session shutdown", async () => { + const first = createHarness(); + await first.emit("session_start", { reason: "startup" }); + first.setIdle(true); + await enqueue(first, "followUp", "/reload"); + await enqueue(first, "followUp", "before shutdown"); + await first.emit("agent_settled"); + await waitFor(() => first.submitted.length === 1); + + await enqueue(first, "followUp", "arrived after scheduling"); + await first.emit("session_shutdown", { reason: "reload" }); + + const second = createHarness(); + second.setIdle(true); + await second.emit("session_start", { reason: "reload" }); + await waitFor(() => second.sent.length === 1); + assert.equal(second.sent[0]?.content, "before shutdown"); + await second.emit("agent_settled"); + assert.equal(second.sent[1]?.content, "arrived after scheduling"); + assert.match(second.notifications[0]?.message ?? "", /Restored 2 queued rows after reload/); + await second.emit("session_shutdown", { reason: "quit" }); +}); + +test("preserves a paused queue and attachments across direct runtime reload", async () => { + const image: ImageContent = { type: "image", data: "AA==", mimeType: "image/png" }; + const first = createHarness(); + await first.emit("session_start", { reason: "startup" }); + await first.emit("input", { + source: "interactive", + text: "paused image row", + images: [image], + streamingBehavior: "followUp", + }); + first.editor.handleInput("escape"); + assert.equal(first.aborted, true); + await first.emit("session_shutdown", { reason: "reload" }); + + const second = createHarness(); + second.setIdle(true); + await second.emit("session_start", { reason: "reload" }); + await new Promise((resolve) => setTimeout(resolve, 0)); + assert.equal(second.sent.length, 0); + assert.match(renderWidget(second), /paused/); + + second.editor.handleInput("enter"); + assert.deepEqual(second.sent[0]?.content, [ + { type: "text", text: "paused image row" }, + image, + ]); + await second.emit("session_shutdown", { reason: "quit" }); +}); + +test("survives repeated queued reloads without expiry, reordering, or duplication", async () => { + const reloadCount = 25; + let runtime = createHarness(); + await runtime.emit("session_start", { reason: "startup" }); + runtime.setIdle(true); + for (let index = 0; index < reloadCount; index += 1) { + await enqueue(runtime, "followUp", "/reload"); + } + await enqueue(runtime, "followUp", "after every reload"); + await runtime.emit("agent_settled"); + + for (let index = 0; index < reloadCount; index += 1) { + await waitFor(() => runtime.submitted.length === 1); + assert.deepEqual(runtime.submitted, ["/reload"]); + await runtime.emit("session_shutdown", { reason: "reload" }); + const replacement = createHarness(); + replacement.setIdle(true); + await replacement.emit("session_start", { reason: "reload" }); + runtime = replacement; + } + + await waitFor(() => runtime.sent.length === 1); + assert.deepEqual(runtime.sent.map((item) => item.content), ["after every reload"]); + await runtime.emit("session_shutdown", { reason: "quit" }); +}); + +test("restores the base editor on shutdown so a reloaded runtime cannot retain stale guards", async () => { + const harness = createHarness(); + const baseFactory = harness.editorFactory; + await harness.emit("session_start", { reason: "startup" }); + harness.wrapEditorFactory(); + await harness.emit("agent_start"); + await new Promise((resolve) => setTimeout(resolve, 5)); + assert.notEqual(harness.editorFactory, baseFactory); + + await harness.emit("session_shutdown", { reason: "reload" }); + assert.equal(harness.editorFactory, baseFactory); + harness.editor.onSubmit?.("/reload"); + assert.deepEqual(harness.submitted, ["/reload"]); +}); + +test("cancels a deferred queued reload when another shutdown wins the race", async () => { + const harness = createHarness(); + await harness.emit("session_start", { reason: "startup" }); + harness.setIdle(true); + await enqueue(harness, "followUp", "/reload"); + await harness.emit("agent_settled"); + await harness.emit("session_shutdown", { reason: "quit" }); + await new Promise((resolve) => setTimeout(resolve, 5)); + assert.deepEqual(harness.submitted, []); +}); + +test("expands restored prompt and full Skill rows from the reloaded runtime", async () => { + const dir = mkdtempSync(join(tmpdir(), "pi-queue-reload-resources-")); + const promptPath = join(dir, "review.md"); + const skillPath = join(dir, "SKILL.md"); + writeFileSync(promptPath, "Old prompt $1"); + writeFileSync(skillPath, "---\nname: bro\ndescription: Plain\n---\nSpeak plainly."); + const sourceInfo = (path: string): SlashCommandInfo["sourceInfo"] => ({ + path, + source: "test", + scope: "temporary", + origin: "top-level", + }); + const commands: SlashCommandInfo[] = [ + { name: "review", source: "prompt", sourceInfo: sourceInfo(promptPath) }, + { name: "skill:bro", source: "skill", sourceInfo: sourceInfo(skillPath) }, + ]; + const image: ImageContent = { type: "image", data: "AA==", mimeType: "image/png" }; + try { + const first = createHarness({ commands }); + await first.emit("session_start", { reason: "startup" }); + first.setIdle(true); + await enqueue(first, "followUp", "/reload"); + await first.emit("input", { + source: "interactive", + text: "/review this", + images: [image], + streamingBehavior: "followUp", + }); + await enqueue(first, "followUp", "/skill:bro simplify"); + await first.emit("agent_settled"); + await waitFor(() => first.submitted.length === 1); + await first.emit("session_shutdown", { reason: "reload" }); + + writeFileSync(promptPath, "Reloaded prompt $1"); + const second = createHarness({ commands }); + second.setIdle(true); + await second.emit("session_start", { reason: "reload" }); + await waitFor(() => second.sent.length === 1); + assert.deepEqual(second.sent[0]?.content, [ + { type: "text", text: "Reloaded prompt this" }, + image, + ]); + await second.emit("agent_settled"); + assert.match(String(second.sent[1]?.content), / { "---", "description: Test prompt", "---", - "$1|$2|$@|${3:-fallback}|${@:2:1}", + "$1|$2|$@|${3:-fallback}|${@:2:1}|${ARGUMENTS:-all-default}|${@:-at-default}", ].join("\n")); try { const review = command("review", "prompt", path); - const expected = "first|two words|first two words|fallback|two words"; + const expected = "first|two words|first two words|fallback|two words|first two words|first two words"; assert.equal(expandQueuedInput('/review first "two words"', [review]), expected); assert.equal(expandQueuedInput('/review first\n"two words"', [review]), expected); } finally { @@ -76,6 +76,40 @@ test("leaves messages and unknown slash input unchanged", () => { assert.equal(expandQueuedInput("/unknown with args", []), "/unknown with args"); }); +test("uses defaults for empty all-argument prompt placeholders", () => { + const dir = mkdtempSync(join(tmpdir(), "pi-queue-prompt-defaults-")); + const path = join(dir, "defaults.md"); + writeFileSync(path, "${ARGUMENTS:-all-default}|${@:-at-default}"); + try { + assert.equal(expandQueuedInput("/defaults", [command("defaults", "prompt", path)]), "all-default|at-default"); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test("classifies only native post-compaction TUI submissions", () => { + const extension = command("deploy", "extension", "/extension.ts"); + const prompt = command("review", "prompt", "/review.md"); + assert.equal(queuesDuringCompaction("ordinary message", [extension, prompt]), true); + assert.equal(queuesDuringCompaction("/unknown as text", [extension, prompt]), true); + assert.equal(queuesDuringCompaction("/review now", [extension, prompt]), true); + assert.equal(queuesDuringCompaction("/skill:bro now", [extension, prompt]), true); + assert.equal(queuesDuringCompaction("/deploy prod", [extension, prompt]), false); + assert.equal(queuesDuringCompaction("/model small", [extension, prompt]), false); + assert.equal(queuesDuringCompaction(" /model small ", [extension, prompt]), false); + assert.equal(queuesDuringCompaction("/debug", [extension, prompt]), false); + assert.equal(queuesDuringCompaction("!echo now", [extension, prompt]), false); + assert.equal(queuesDuringCompaction(" !echo now ", [extension, prompt]), false); + assert.equal(queuesDuringCompaction("", [extension, prompt]), false); + assert.equal(queuesDuringCompaction(" ", [extension, prompt]), false); + + assert.equal(queuesDuringCompaction("ordinary follow-up", [extension, prompt], "followUp"), true); + assert.equal(queuesDuringCompaction("/model small", [extension, prompt], "followUp"), true); + assert.equal(queuesDuringCompaction("!echo now", [extension, prompt], "followUp"), true); + assert.equal(queuesDuringCompaction("/deploy prod", [extension, prompt], "followUp"), false); + assert.equal(queuesDuringCompaction(" ", [extension, prompt], "followUp"), false); +}); + test("rejects discovered extension commands", () => { const extension = command("deploy", "extension", "/extension.ts"); assert.throws( diff --git a/test/tui-evidence.sh b/test/tui-evidence.sh new file mode 100755 index 0000000..62ed603 --- /dev/null +++ b/test/tui-evidence.sh @@ -0,0 +1,253 @@ +#!/usr/bin/env bash +set -euo pipefail + +ROOT=$(cd "$(dirname "$0")/.." && pwd) +ARTIFACT_DIR=${1:-"$ROOT/.artifacts/tui-evidence"} +PI_BIN=${PI_BIN:-pi} +STATE_DIR=$(mktemp -d "${TMPDIR:-/tmp}/pi-queue-tui.XXXXXX") +WORK_DIR="$STATE_DIR/workspace" +SOCKET="$STATE_DIR/tmux.sock" +SESSION=pi-queue-evidence +PANE="$SESSION:0.0" +mkdir -p "$ARTIFACT_DIR" "$WORK_DIR/.pi" + +cleanup() { + tmux -S "$SOCKET" kill-server >/dev/null 2>&1 || true + rm -rf "$STATE_DIR" +} +trap cleanup EXIT + +write_settings() { + local mode=$1 + cat > "$WORK_DIR/.pi/settings.json" < "$ARTIFACT_DIR/$name.txt" +} + +wait_screen() { + local needle=$1 + local timeout=${2:-20} + local deadline=$((SECONDS + timeout)) + while (( SECONDS < deadline )); do + if tmux -S "$SOCKET" capture-pane -p -J -t "$PANE" -S -300 | grep -Fq -- "$needle"; then + return 0 + fi + sleep 0.1 + done + echo "Timed out waiting for screen text: $needle" >&2 + tmux -S "$SOCKET" capture-pane -p -J -t "$PANE" -S -120 >&2 + return 1 +} + +wait_file() { + local file=$1 + local needle=$2 + local timeout=${3:-20} + local deadline=$((SECONDS + timeout)) + while (( SECONDS < deadline )); do + if [[ -f "$file" ]] && grep -Fq -- "$needle" "$file"; then + return 0 + fi + sleep 0.1 + done + echo "Timed out waiting for $needle in $file" >&2 + [[ -f "$file" ]] && tail -20 "$file" >&2 + return 1 +} + +wait_line_count() { + local file=$1 + local expected=$2 + local timeout=${3:-20} + local deadline=$((SECONDS + timeout)) + while (( SECONDS < deadline )); do + local count=0 + [[ -f "$file" ]] && count=$(wc -l < "$file") + if (( count >= expected )); then + return 0 + fi + sleep 0.1 + done + echo "Timed out waiting for $expected lines in $file" >&2 + return 1 +} + +send_text() { + tmux -S "$SOCKET" send-keys -t "$PANE" -l -- "$1" + tmux -S "$SOCKET" send-keys -t "$PANE" Enter +} + +queue_follow_up() { + tmux -S "$SOCKET" send-keys -t "$PANE" -l -- "$1" + tmux -S "$SOCKET" send-keys -t "$PANE" -l -- $'\e[13;3u' + sleep 0.1 +} + +start_pi() { + local context_window=$1 + local mode=$2 + write_settings "$mode" + tmux -S "$SOCKET" kill-session -t "$SESSION" >/dev/null 2>&1 || true + rm -f "$STATE_DIR"/{events.jsonl,provider-calls.jsonl,runtime-inits.log,gate-*,hold-summary,release-summary,fail-summary} + tmux -S "$SOCKET" -f /dev/null new-session -d -s "$SESSION" -x 120 -y 36 -c "$WORK_DIR" + tmux -S "$SOCKET" set-option -g extended-keys on + tmux -S "$SOCKET" set-option -g extended-keys-format csi-u + local launch + printf -v launch \ + 'PI_QUEUE_TUI_STATE_DIR=%q PI_QUEUE_TUI_CONTEXT_WINDOW=%q %q --no-session --no-extensions --no-skills --no-prompt-templates --no-context-files --approve --model faux/queue-e2e --tools bash -e %q -e %q --skill %q --prompt-template %q' \ + "$STATE_DIR" "$context_window" "$PI_BIN" \ + "$ROOT/test/fixtures/tui-faux-provider.ts" "$ROOT/index.ts" \ + "$ROOT/test/fixtures/skills" "$ROOT/test/fixtures/prompts" + tmux -S "$SOCKET" send-keys -t "$PANE" -l -- "$launch" + tmux -S "$SOCKET" send-keys -t "$PANE" Enter + wait_screen "[Extensions]" 20 +} + +# Manual compaction success/failure, abort recovery, repeated reload, and resources. +echo "Running manual/reload/resource scenario" +start_pi 100000 one-at-a-time +send_text "manual seed one" +wait_screen "FAUX RESPONSE: manual seed one" +send_text "manual seed two" +wait_screen "FAUX RESPONSE: manual seed two" +send_text "BLOCK:success" +wait_file "$STATE_DIR/provider-calls.jsonl" "BLOCK:success" +queue_follow_up "/compact preserve evidence" +queue_follow_up "after manual compaction" +touch "$STATE_DIR/gate-success" +wait_screen "FAUX RESPONSE: after manual compaction" 30 +wait_file "$STATE_DIR/events.jsonl" '"reason":"manual"' + +send_text "BLOCK:failure" +wait_file "$STATE_DIR/provider-calls.jsonl" "BLOCK:failure" +queue_follow_up "/compact fail evidence" +queue_follow_up "after failed compaction" +touch "$STATE_DIR/fail-summary" "$STATE_DIR/gate-failure" +wait_screen "FAUX RESPONSE: after failed compaction" 30 +wait_screen "synthetic TUI summary failure" +rm -f "$STATE_DIR/fail-summary" + +send_text "BLOCK:abort" +wait_file "$STATE_DIR/provider-calls.jsonl" "BLOCK:abort" +queue_follow_up "after abort resume" +tmux -S "$SOCKET" send-keys -t "$PANE" Escape +wait_screen "paused" +capture_plain "abort-paused" +tmux -S "$SOCKET" send-keys -t "$PANE" Enter +wait_screen "FAUX RESPONSE: after abort resume" 20 + +send_text "BLOCK:reloads" +wait_file "$STATE_DIR/provider-calls.jsonl" "BLOCK:reloads" +queue_follow_up "/reload" +queue_follow_up "/reload" +queue_follow_up "after repeated reload" +touch "$STATE_DIR/gate-reloads" +wait_screen "FAUX RESPONSE: after repeated reload" 30 +wait_line_count "$STATE_DIR/runtime-inits.log" 3 +cp "$STATE_DIR/runtime-inits.log" "$ARTIFACT_DIR/reload-runtime-inits.log" + +send_text "BLOCK:expansions" +wait_file "$STATE_DIR/provider-calls.jsonl" "BLOCK:expansions" +queue_follow_up "/review alpha beta" +queue_follow_up "/skill:bro gamma" +touch "$STATE_DIR/gate-expansions" +wait_screen 'FAUX RESPONSE: &2 + exit 1 +fi +ordinary_line=$(grep -nF "FAUX RESPONSE: ordinary native during compaction" "$ARTIFACT_DIR/native-before-command.txt" | tail -1 | cut -d: -f1) +reload_line=$(grep -nF "Reloaded keybindings" "$ARTIFACT_DIR/native-before-command.txt" | tail -1 | cut -d: -f1) +if (( ordinary_line >= reload_line )); then + echo "Reload executed before native post-compaction input" >&2 + exit 1 +fi + +# Real public overflow path with a tiny faux context window. +echo "Running automatic overflow scenario" +start_pi 1000 one-at-a-time +printf 'BLOCK:overflow ' > "$STATE_DIR/overflow-input.txt" +head -c 20000 /dev/zero | tr '\0' x >> "$STATE_DIR/overflow-input.txt" +tmux -S "$SOCKET" load-buffer "$STATE_DIR/overflow-input.txt" +tmux -S "$SOCKET" paste-buffer -d -t "$PANE" +tmux -S "$SOCKET" send-keys -t "$PANE" Enter +wait_file "$STATE_DIR/provider-calls.jsonl" "BLOCK:overflow" +queue_follow_up "after automatic overflow" +touch "$STATE_DIR/gate-overflow" +wait_screen "FAUX RESPONSE: after automatic overflow" 40 +wait_file "$STATE_DIR/events.jsonl" '"reason":"overflow"' +capture_plain "automatic-overflow" +cp "$STATE_DIR/events.jsonl" "$ARTIFACT_DIR/overflow-events.jsonl" +cp "$STATE_DIR/provider-calls.jsonl" "$ARTIFACT_DIR/overflow-provider-calls.jsonl" +overflow_follow_up_count=$(grep -Fc '"prefix":"after automatic overflow"' "$ARTIFACT_DIR/overflow-provider-calls.jsonl" || true) +if (( overflow_follow_up_count != 1 )); then + echo "Automatic-overflow follow-up completed $overflow_follow_up_count times, expected exactly once" >&2 + exit 1 +fi + +# Pi's all-mode setting delivers the whole visible lane in FIFO order. +echo "Running all-mode scenario" +start_pi 100000 all +send_text "BLOCK:allmode" +wait_file "$STATE_DIR/provider-calls.jsonl" "BLOCK:allmode" +queue_follow_up "all row one" +queue_follow_up "all row two" +queue_follow_up "all row three" +touch "$STATE_DIR/gate-allmode" +wait_screen "FAUX RESPONSE: all row three" 30 +capture_plain "all-mode" +cp "$STATE_DIR/provider-calls.jsonl" "$ARTIFACT_DIR/all-mode-provider-calls.jsonl" +all_mode_context_count=$( + grep -Fc '"userPrefixes":["BLOCK:allmode","all row one","all row two","all row three"]' \ + "$ARTIFACT_DIR/all-mode-provider-calls.jsonl" \ + || true +) +if (( all_mode_context_count != 1 )); then + echo "All-mode rows did not reach one provider context exactly once in FIFO order" >&2 + exit 1 +fi + +working_tree=clean +if [[ -n "$(git -C "$ROOT" status --porcelain --untracked-files=all)" ]]; then + working_tree=dirty +fi +{ + echo "pi: $($PI_BIN --version)" + echo "commit: $(git -C "$ROOT" rev-parse HEAD)" + echo "working tree: $working_tree" + echo "manual events: $(tr '\n' ' ' < "$ARTIFACT_DIR/manual-events.jsonl")" + echo "overflow events: $(tr '\n' ' ' < "$ARTIFACT_DIR/overflow-events.jsonl")" + echo "runtime initializations across two queued reloads: $(wc -l < "$ARTIFACT_DIR/reload-runtime-inits.log")" + echo "captures: abort-paused, manual-reload-resources, native-before-command, automatic-overflow, all-mode" +} > "$ARTIFACT_DIR/summary.txt" +cat "$ARTIFACT_DIR/summary.txt"