WIP / POC: AI Transport interactive TUI demo#365
Draft
mattheworiordan wants to merge 17 commits intomainfrom
Draft
WIP / POC: AI Transport interactive TUI demo#365mattheworiordan wants to merge 17 commits intomainfrom
mattheworiordan wants to merge 17 commits intomainfrom
Conversation
Add @ably/ai-transport SDK, Ink TUI framework, and command topic structure for interactive AI Transport demos. The streaming demo command is a placeholder with --role, --channel, --endpoint, and --auth-endpoint flags. Bump ably to ^2.21.0 for SDK compatibility, move get-port to runtime dependencies. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Core library modules for AI Transport demos: - Fake LLM stream generator with bursty pacing and abort support - Demo codec implementing full @ably/ai-transport Codec interface using createEncoderCore/createDecoderCore helpers - Mutable messages detection via Control API and data plane fallback (error 93002) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Local HTTP server with auto-port finding and presence announcement. Client discovery subscribes to presence, finds the server endpoint, with timeout and graceful fallback for both existing members and late-arriving servers. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Split-pane terminal UI with Ink (React for CLIs): - ClientPanel: conversation display with streaming cursor - ServerPanel: timestamped server activity log - DebugPanel: collapsible transport-level event console - InputBar: text input with context-sensitive hints - App: root layout with role-based rendering Move lib/ and ui/ from commands/ to services/ai-transport-demo/ to prevent oclif from treating them as commands. Add ESLint config for TSX files with React plugin support. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Move debug console and input bar inside the client panel border for a clearer client/server separation - Debug events always visible (2 lines default, Tab expands to 8) - Input bar with Claude Code-style prompt inside client panel - Server panel as standalone log with space for future controls - Fix Ink not rendering due to CI_* env var triggering is-in-ci detection — dynamically import ink after setting CI=0 - Use round border style for cleaner panel appearance Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Orchestrator connects server transport, client transport, HTTP server, and fake LLM via the @ably/ai-transport SDK - useDemo hook connects orchestrator events to React UI state - Graceful mutable messages (93002) error handling: detects the error, closes Ably connection to stop NACK flood, shows setup instructions - Fix duplicate user messages (don't re-publish from server) - Fix turn.end() crash after connection close Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Outputs just the raw API key value with no formatting, making it easy to use in shell scripts and environment variables: ABLY_API_KEY=$(ably auth keys current --value-only) node my-app.js Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When mutableMessages is not enabled, the AIT encoder floods stderr with [AblySDK Error] NACK logs and PromiseRejectionHandledWarnings. These are a known SDK issue (reported separately) but we need the demo to handle it gracefully. Intercept console.error/console.log and process warnings to suppress the NACK flood once the mutable messages error is detected. The TUI error panel shows the clean setup instructions instead. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Role modes: --role client/server show appropriate single-panel views - Server-only mode shows channel name + connect command for other terminal - serverNotFound event when presence discovery times out - serverStatus tracked properly in useDemo hook - Mutable messages error: auto-exits after rendering error panel using useEffect (no setTimeout), message stays on screen - Fix error leak after exit: keep stdout/stderr interceptors active through entire cleanup lifecycle, only restore in class finally() after Ably client is fully closed Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Reject --channel without a namespace (no colon) — can't suggest a rule to create without knowing the namespace - Make mutable messages error dialog dynamic — shows the actual namespace from the channel name, not hardcoded "ai-demo" - Default auto-generated channels still use ai-demo: namespace Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…events The demo's createEncoder was dropping options.extras.headers and options.onMessage when wrapping createEncoderCore. The server transport passes transport-level headers (x-ably-turn-id, x-ably-role, x-ably-msg-id) via options.extras.headers, and without them the client's stream router could not correlate incoming events to active turns — the ActiveTurn.stream never received any events and the UI saw no streaming response. Add regression tests that fail if the encoder stops forwarding headers or the onMessage hook. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Ably does not immediately remove presence entries when a demo server crashes uncleanly, so stale http endpoints linger on the channel. Previously the client picked the first entry (or the newest by timestamp) and happily POSTed to a dead server. Sort candidates by timestamp newest-first and HTTP-probe each with an OPTIONS request (750ms timeout) before accepting. If none respond, subscribe to presence and probe each new 'enter'/'present' event so late-arriving live servers supersede stale ones. Add tests covering: probe rejection of stale entries, fallback to live entry, and the client-first scenario where only stale entries exist until a real server starts. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Complete the end-to-end streaming demo with the fixes that make it usable
in practice.
Message ordering
Publish user messages via turn.addMessages() on the server so they land
on the channel with a real serial. The client transport's send() only
does an optimistic local insert; without the server also publishing,
user messages stayed at null-serial and sorted AFTER serial-bearing
assistant responses in ConversationTree.flatten(). The client
de-duplicates via its _ownMsgIds set keyed on x-ably-msg-id, so no
double-render.
Server log: in-place token progress with turn IDs
New "serverStatus" event on the orchestrator carries a transient
single-line status. Every text-delta emits "<turnId>… streaming: N
tokens" which updates in place rather than appending a line per token.
Cleared on turn end or error; the final "turn:end N tokens" line then
appends to history. All lifecycle logs (turn:start, streaming response,
turn:end, failures) now lead with a short turn ID so you can correlate
them.
Conversation viewport + scrollback
ClientPanel rewritten to slice messages into a bounded viewport with
independent head/tail crop. Required because Ink does not clip flex
overflow — content taller than the client panel would push the debug
log and input bar off the bottom. selectWindow() computes absolute
line ranges per message and head-crops content above the window,
tail-crops content below, so the right slice renders whether the
window is at the top of history, middle, or pinned to live bottom.
PageUp/PageDown (Fn+↑/↓ on Mac) scroll by ~90% of a screen with one
line of overlap. Home/End (Fn+←/→) jump to top and back to live.
Sending a new message auto-resets to live. When new tokens arrive
while scrolled up, the offset is adjusted so the view stays anchored
to the absolute content the user was reading rather than drifting
with the stream. "↓ N lines below · End to jump to live" and
"↑ N lines above" hints show at the viewport edges.
Misc UI fixes
* Every message (not just assistant) gets a blank separator line above
it so "You:" doesn't butt up against the previous "Agent:" response.
* Drop the 🖥 emoji from the server panel header — it renders as two
terminal columns but Ink measures it as one, which crowded the
"Server" label and shifted the right border by one column.
* Revert fake-LLM baseDelayMs from 80ms back to 40ms — the comment
about rate limits was wrong; appendMessage is not rate-limited at
the 50/sec per-channel publish ceiling.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Introduce the second demo in the ai-transport topic. Type a message while
the agent is still streaming and the in-flight response is cancelled,
marked interrupted in the UI, and a new turn starts for the new message.
Implementation
* New command src/commands/ai-transport/demo/barge-in.ts, plus
response bank responses/barge-in.ts full of deliberately long,
self-aware replies that invite the user to interrupt.
* Shared runner lib/run-demo.ts extracted from the old streaming.ts
so barge-in (and the future cancel demo) stay as thin command
shells. streaming.ts re-pointed at the shared runner.
* orchestrator.sendMessage() detects an in-flight own turn and
publishes clientTransport.cancel({own: true}) before starting the
new turn. The currently-streaming assistant message id is added to
interruptedMessageIds, and emitMessages() passes both `streaming`
and `interrupted` flags through to the UI.
* use-demo.ts hook forwards the new `interrupted` flag. ClientPanel
already renders "[interrupted]" in warning colour.
README regenerated.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
The barge-in demo published cancel on the client side and the SDK
aborted the server's turn, but fake-LLM kept pumping tokens: I was
passing my own locally-created AbortController.signal into the
generator, which the SDK has no way to reach. The result was two
turns appearing to stream in parallel on the server log after a
barge-in (the newer one plus the supposedly-cancelled first one).
Use turn.abortSignal (exposed as a getter by the AIT SDK's Turn)
directly. When the server transport receives a matching cancel
event on the channel, it fires that signal, which our fake-LLM
already respects. The duplicate activeTurnAbort state is now
unnecessary and has been removed. cancelActiveTurn() likewise
just publishes clientTransport.cancel({own: true}) and lets the
signal flow back.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…l matching
Barge-in cancels were silently no-ops: the Ably connection's clientId
defaults to 'ably-cli-XYZ' (set by ControlBaseCommand.setClientId), but
the orchestrator's clientId was a separate 'demo-XYZ' fallback. When
the client published cancel({own: true}), the server transport's filter
compared the cancel message's publisher clientId (the connection
clientId) against the turn's registered clientId (the demo clientId)
and found no matches — so no turns were aborted, the turn's abortSignal
never fired, fake-LLM kept generating, and streamResponse never
returned. The server log showed both turns' lifecycle events but
turn:end never appeared for the first turn.
Prefer the Ably connection's clientId when constructing the demo
clientId. Fall back to --client-id flag or the random demo-XYZ only
when the connection has no clientId set (e.g. `--client-id none`).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR is a proof of concept I'm prototyping while working on AI Transport docs improvements. It is not intended to merge. It exists so the docs team has a branch to reference while we figure out how to showcase AIT capabilities to developers.
The idea
An interactive TUI (terminal UI) demo that can be spun up via the CLI (and eventually the web CLI) to experience AI Transport features end-to-end in a few keystrokes, rather than reading them on a page. The demo uses the real
@ably/ai-transportSDK against real Ably infrastructure, with a fake LLM standing in for the model so it runs without an API key from any provider.Split-pane TUI: client on the left (conversation + input + debug), server on the right (turn lifecycle, token progress, turn IDs). Both roles can run in one terminal (
--role both) or split across two terminals on the same Ably channel (--role server/--role client) so you can watch the protocol flow live.What's in this branch so far
Two demos are working end-to-end:
ably ai-transport demo streaming— the baseline. Type a message, see tokens stream back through real Ably channels using the AIT SDK's durable streaming, with a visible turn lifecycle and in-place token-count status on the server panel.ably ai-transport demo barge-in— the showstopper. Start asking something, then interrupt mid-stream by typing a new message. The in-flight response is cancelled, marked[interrupted]in the UI, and a new turn starts immediately — no waiting for the LLM to finish.Plus a handful of things the TUI needs to feel right:
Not yet in
Screenshot / GIF
Why it's a draft
src/services/ai-transport-demo/lib/run-demo.tsis a deliberate minimal abstraction; it may get refactored once the cancel demo lands and we know what's truly shared.Try it locally
git fetch origin feature/ai-transport-demo git checkout feature/ai-transport-demo pnpm install pnpm prepare ./bin/run.js ai-transport demo streaming # or: barge-inYou'll need an Ably app with a channel rule on the
ai-demo:*namespace with Mutable Messages enabled. The CLI explains this with a clear error dialog if the rule isn't there.