Proposal: flexible budget and limit control across session continuation

[aheritier]

Status

Proposal only — this issue requests design and implementation planning; it does not authorize an implementation yet.

Why

Today, a run can terminate when its iteration or runtime budget is exhausted, but the user experience for restoring and continuing is poor: the continuation path can be unclear, and restarting or forking can make accounting look reset or incomplete. That creates trust concerns about whether a resumed session is still honoring the original limits, especially for unattended, shared, or security-sensitive workloads. A session should be able to continue deliberately without silently creating a fresh allowance or bypassing a prior stop reason.

This builds on the budget discussion in #3701, but focuses on durable accounting and truthful continuation semantics rather than only declaring limits.

What

Design a persisted, root-owned budget ledger for each session tree:

• Persist cumulative usage and limit decisions at the session root; child sessions and forks must have explicit, auditable ownership and accounting semantics.
• Make restore and fork behavior truthful and deterministic. Restoring continues the same ledger; forking must clearly define whether it shares, snapshots, or starts a separately authorized ledger, with no accidental reset or double counting.
• Support policy-governed additive extensions (for example, an explicit user/API/CLI approval to add budget), recording who/what authorized the extension, when, the amount, and the resulting limit. Extensions must not be an implicit consequence of restore, retry, or fork.
• Keep distinct concepts distinct: hard safety limits, spend/time/token budgets, and iteration limits should have separate configuration, enforcement, stop reasons, and user-facing accounting.
• Provide deliberate continuation from the TUI, API, and CLI, including inspection of remaining/exhausted budget, the stop reason, ledger identity, and any extension history. Headless/API behavior must be deterministic and machine-readable.
• Treat the ledger as security-sensitive state: validate persisted data, use atomic/concurrency-safe updates, prevent unauthorized extension or path-based substitution, and fail closed or visibly degraded when accounting integrity cannot be established.

How / proposed phased rollout

1. Semantics and threat model: define the ledger schema, root/session/fork identity, accounting units, hard-limit vs budget vs iteration-limit precedence, authorization model, crash/concurrency behavior, and compatibility contract. Use the shared detailed plan flexible-budget-limit-control as the authoritative source for full PR sequencing and design decisions.
2. Ledger foundation: add versioned persistence, migration handling, integrity/atomicity safeguards, and runtime accounting with focused tests. Preserve current behavior for configurations that do not opt into the new model.
3. Enforcement and continuation: implement truthful restore/fork semantics and policy-checked additive extensions, with explicit stop reasons and audit records. Add API/CLI commands and TUI flows only after the core state machine is stable.
4. Compatibility and rollout gate: document migration and backward-compatibility behavior, validate old sessions/configurations, add failure-injection/concurrency/security tests, and gate release behind opt-in/feature rollout until ledger correctness and continuation UX are verified against the detailed plan.

Acceptance criteria

• A resumed session cannot silently reset or exceed its root-owned accounting.
• Fork behavior and accounting ownership are visible and covered by tests.
• Budget extensions require the configured policy/authorization and leave an auditable record.
• Hard limits, budgets, and iteration limits remain independently inspectable and enforceable.
• TUI, API, and CLI expose consistent continuation and machine-readable outcomes.
• Existing sessions and configurations have a documented, tested migration/backward-compatibility path.

The detailed implementation plan, including full PR sequencing, migration, safety, backwards compatibility, validation, and rollout gate, is shared under slug flexible-budget-limit-control.

[priyanka25aug]

This proposal maps closely to failure-budget accounting patterns I've worked with in production LLM systems, so a few design observations that may be useful for the semantics phase (step 1).

Budget states should be graduated, not binary. A ledger that only knows "within budget" vs "exhausted" pushes every continuation decision to the moment of exhaustion — which is exactly when the user has the least context and the strongest incentive to blindly extend. In my failure-budget work I've found four utilisation states (healthy / elevated ≥50% / warning ≥80% / breached ≥100%) change behavior meaningfully: warning-level is where a human decides deliberately whether to extend, and headless callers can subscribe to the threshold-crossing events rather than polling. Since the ledger is already root-owned and persisted, emitting state transitions as auditable ledger entries (not just the extension events) gives you the "truthful accounting" property for free — a resumed session's state history shows when it got hot, not just that it died.

Keep the risk class of the workload separate from the size of the budget. Two sessions with the same $5 budget can warrant very different exhaustion behavior: an unattended CI eval should fail closed, while an interactive session should stop-and-ask. In the taxonomy I published for production LLM failures (Beyond Hallucination), I classify use cases into failure-budget classes (FC-A decision-critical → FC-D experimental) precisely because the policy at the limit — not the limit itself — is what differs by deployment context. The proposal's separation of hard safety limits from spend budgets already gestures at this; I'd suggest making the exhaustion policy (fail-closed / stop-and-ask / degrade-visibly) a first-class field on the ledger rather than deriving it from which limit tripped, so forks inherit policy explicitly and audit records can show that a policy — not only an amount — was authorized.

Weighted accounting deserves a slot in the schema even if v1 doesn't use it. Cost, tokens, and working time (#3701's dimensions) accumulate at different rates and units, and my experience is that teams eventually want severity- or dimension-weighted utilisation ("this session is 40% through cost but 95% through time"). If each dimension is a separate ledger column with its own state, the "distinct concepts stay distinct" acceptance criterion falls out naturally, and per-dimension stop reasons stay machine-readable.

One sharp edge for the fork semantics: snapshot-at-fork plus separately-authorized top-ups is the only variant I've seen survive contact with concurrent children. Shared-ledger forks create write contention and make "who spent this" unanswerable in the audit trail; reset-at-fork is the accidental-double-budget failure mode the proposal already rules out. A snapshot with an explicit parent-ledger reference keeps lineage queryable ("total spend of this session tree") without shared mutable state.

Happy to expand any of this — the failure-budget calculator in the repo above implements the graduated states and weighted accounting if a concrete reference is useful.