feat(tokens): per-turn token tracking and StreamUsageSummary event#219
Open
torrmal wants to merge 4 commits into
Open
feat(tokens): per-turn token tracking and StreamUsageSummary event#219torrmal wants to merge 4 commits into
torrmal wants to merge 4 commits into
Conversation
Add StreamUsageSummary event type to provider.py. In session.py, accumulate input/output/cache tokens across every LLM call in a turn, log them at INFO level, and yield a StreamUsageSummary after the turn loop completes. Also pre-merge turn_stream() signature additions from PR #218 (trace_tags / trace_metadata / TraceContext kwargs) to avoid a conflict when that PR lands. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…k artifacts The ~5,200-token backend template (FastAPI canonical template, 7-step workflow, deployment rules) was unconditionally included in every system prompt — even for simple chat turns, data analysis, and any session with no backend artifacts. prompt_builder.py: add include_backend_prompt: bool = False to build(). session.py: add _has_fullstack_artifacts() which scans the workspace ArtifactStore for any fullstack-stateless-app / fullstack-stateful-app artifact. Pass the result as include_backend_prompt on every prompt build. Because _build_system_prompt() is called per-LLM-call (not per-turn), the prompt flips to include the template on the very next call after create_artifact() registers a fullstack artifact — no turns are lost. Sessions with no fullstack artifacts save ~5,200 tokens per turn. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ntent detection _has_fullstack_artifacts() only checked existing workspace artifacts, missing the case where a user asks to build a backend app from scratch in a fresh session — the model would enter the first LLM call without the FastAPI template and deployment rules. Replace with _needs_backend_prompt(user_message) which fires on any of: 1. Workspace already contains a fullstack-stateless/stateful artifact. 2. Current user message contains backend/app-building intent keywords. 3. Any prior user turn in conversation history matched (2) — covers multi-turn sessions where the original request was several turns ago. _message_suggests_backend() is a pure keyword heuristic (no LLM call): strong signals (fastapi, flask, backend, fullstack, etc.) trigger alone; weaker signals (app, server, service, ...) require a co-occurring build verb. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
StreamUsageSummarydataclass toprovider.py— carriesinput_tokens,output_tokens,cache_read_input_tokens,cache_creation_input_tokensfor a full turnsession.py, accumulates token counts across every LLM call in a turn (main loop + error-fallback path), logs them at INFO level, and yields aStreamUsageSummaryafter the turn loop completesturn_stream()signature additions from PR send trace data #218 (trace_tags/trace_metadata/TraceContextkwargs) to avoid a conflict when that landsLog line emitted per turn
Cache fields will be non-zero once the Anthropic prompt-caching work (stashed on this branch) is wired in.
Test plan
turn_tokenslog line appears with non-zeroin/out🤖 Generated with Claude Code