feat: add --toon output format for token-optimized CLI output - #215
feat: add --toon output format for token-optimized CLI output#215akb4q wants to merge 5 commits into
Conversation
Add a new global --toon flag that outputs command results in TOON format (https://toonformat.dev), a token-optimized notation designed for LLM consumption — 6-9x more compact than JSON or human-readable tables. Implementation: - toonEscapeValue / toonEscapeKey: spec-compliant quoting (§7.1-7.2) and key encoding (§7.3) - outputToon: tabular array format for list commands - outputJsonToon: tabular for uniform arrays, key-value for single objects - Module-level flag set via Commander preAction hook - outputSuccess/outputInfo suppressed in TOON mode (non-structured noise) Changes: src/index.ts — register --toon global option + preAction hook src/lib/output.ts — add TOON formatters + route outputTable/outputJson Token savings measured on insforge projects list: TOON 129B vs JSON 989B (7.7x), vs human table 1162B (9x). Tested: build (tsup), 649 unit tests (1 pre-existing cloudflare failure), integration on projects list, whoami, functions list, orgs list.
|
Thanks for the PR, @akb4q! A quick note on our workflow: we ask contributors to open an issue first, get it assigned, then submit a PR that links it (e.g. "Closes #123"). This PR isn't linked to any issue. It'll still be reviewed, but please open an issue and claim it (comment that you'd like it assigned to you) so the work is tracked. |
agent-zhang-beihai
Bot
added
the
needs-issue
WalkthroughThe CLI adds a global ChangesTOON output mode
Estimated code review effort: 3 (Moderate) | ~20 minutes Possibly related issues
Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Greptile SummaryAdds a global TOON output mode.
Confidence Score: 4/5The PR is not yet safe to merge because The global hook enables TOON for login and structured branching recognizes the flag, but login serializes results directly with JSON.stringify and bypasses the TOON-aware output helper. Files Needing Attention: src/index.ts and src/commands/login.ts
|
| Filename | Overview |
|---|---|
| src/index.ts | Registers global TOON mode, but applying it to every action exposes login paths that bypass the TOON-aware output helper. |
| src/lib/errors.ts | Treats --toon as structured output so command handlers generally select their existing JSON-data branches. |
| src/lib/output.ts | Adds TOON encoding for shared JSON and table output and suppresses unstructured status messages. |
| src/lib/output.test.ts | Covers scalar escaping, nested JSON encoding, message suppression, and tabular TOON output. |
| package.json | Adds the TOON encoder runtime dependency. |
| package-lock.json | Locks the TOON encoder dependency to version 4.1.0 with integrity metadata. |
Comments Outside Diff (1)
-
src/index.ts, line 165-169 (link)Login bypasses TOON encoding
When a non-interactive login flow runs with
--toon, the global hook enables TOON mode and the command selects structured output, but the login handlers serialize their results directly withJSON.stringifyinstead ofoutputJson, causing consumers expecting TOON to receive JSON.
Reviews (5): Last reviewed commit: "Merge branch 'main' into feat/toon-outpu..." | Re-trigger Greptile
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/lib/output.ts`:
- Around line 17-32: Resolve the ESLint violations in the quote-formatting logic
around needsQuoting: replace the escaped opening bracket in the punctuation
regex with a literal bracket, and remove both control-character regexes in favor
of equivalent character-code checks and replacement handling. Preserve the
existing quoting and escaping behavior, including Unicode escapes for
unsupported control characters.
- Around line 99-124: Replace the ad hoc serialization in outputJsonToon with a
conformant recursive TOON encoder, preferably using the official
`@toon-format/toon` encode API. Preserve heterogeneous array values and nulls,
recursively encode nested objects and arrays rather than stringifying them, and
retain null fields/entries; add decode round-trip coverage for these cases.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 17e2f641-7d66-42fc-a61f-608a2ac82f7b
📒 Files selected for processing (2)
src/index.tssrc/lib/output.ts
| const needsQuoting = | ||
| s === '' || | ||
| s === 'true' || s === 'false' || s === 'null' || | ||
| /^[+-]?[0-9]+(\.[0-9]+)?([eE][+-]?[0-9]+)?$/.test(s) || | ||
| /^[ \t]|[ \t]$/.test(s) || | ||
| /^[-#]/.test(s) || | ||
| /[:,"\\{}\[\]]/.test(s) || | ||
| /[\x00-\x1f]/.test(s); | ||
| if (!needsQuoting) return s; | ||
| return '"' + s | ||
| .replace(/\\/g, '\\\\') | ||
| .replace(/"/g, '\\"') | ||
| .replace(/\n/g, '\\n') | ||
| .replace(/\r/g, '\\r') | ||
| .replace(/\t/g, '\\t') | ||
| .replace(/[\x00-\x08\x0b\x0c\x0e-\x1f]/g, c => '\\u' + c.charCodeAt(0).toString(16).padStart(4, '0')) + '"'; |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟠 Major | ⚡ Quick win
Resolve the new ESLint errors.
The escaped [ and both control-character regexes are reported as ESLint errors, leaving this changed file failing lint. Replace the control-character regex checks/replacement with character-code handling and use a literal [.
🧰 Tools
🪛 ESLint
[error] 23-23: Unnecessary escape character: [.
(no-useless-escape)
[error] 24-24: Unexpected control character(s) in regular expression: \x00, \x1f.
(no-control-regex)
[error] 32-32: Unexpected control character(s) in regular expression: \x00, \x08, \x0b, \x0c, \x0e, \x1f.
(no-control-regex)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/lib/output.ts` around lines 17 - 32, Resolve the ESLint violations in the
quote-formatting logic around needsQuoting: replace the escaped opening bracket
in the punctuation regex with a literal bracket, and remove both
control-character regexes in favor of equivalent character-code checks and
replacement handling. Preserve the existing quoting and escaping behavior,
including Unicode escapes for unsupported control characters.
Source: Linters/SAST tools
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
1 issue found across 2 files
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="src/index.ts">
<violation number="1" location="src/index.ts:119">
P1: Using `--toon` alone can produce an empty, human-readable, or raw response instead of TOON. The new flag is parsed by the preAction hook, but command handlers still branch only on `getRootOpts(cmd).json`: `whoami --toon` takes its `outputInfo` branch and both messages are then suppressed, while commands such as `functions invoke` and `metadata` write directly with `console.log` and bypass the TOON renderer. This makes the advertised global format inconsistent unless callers already know to add `--json`; treating TOON as an effective structured-output mode (or explicitly rejecting/documenting it without `--json`) would avoid these silent/mixed responses.</violation>
</file>
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
| // Global options | ||
| program | ||
| .option('--json', 'Output in JSON format') | ||
| .option('--toon', 'Output in TOON format (token-optimized notation, see toonformat.dev)') |
There was a problem hiding this comment.
P1: Using --toon alone can produce an empty, human-readable, or raw response instead of TOON. The new flag is parsed by the preAction hook, but command handlers still branch only on getRootOpts(cmd).json: whoami --toon takes its outputInfo branch and both messages are then suppressed, while commands such as functions invoke and metadata write directly with console.log and bypass the TOON renderer. This makes the advertised global format inconsistent unless callers already know to add --json; treating TOON as an effective structured-output mode (or explicitly rejecting/documenting it without --json) would avoid these silent/mixed responses.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At src/index.ts, line 119:
<comment>Using `--toon` alone can produce an empty, human-readable, or raw response instead of TOON. The new flag is parsed by the preAction hook, but command handlers still branch only on `getRootOpts(cmd).json`: `whoami --toon` takes its `outputInfo` branch and both messages are then suppressed, while commands such as `functions invoke` and `metadata` write directly with `console.log` and bypass the TOON renderer. This makes the advertised global format inconsistent unless callers already know to add `--json`; treating TOON as an effective structured-output mode (or explicitly rejecting/documenting it without `--json`) would avoid these silent/mixed responses.</comment>
<file context>
@@ -116,6 +116,7 @@ program
// Global options
program
.option('--json', 'Output in JSON format')
+ .option('--toon', 'Output in TOON format (token-optimized notation, see toonformat.dev)')
.option('--forger', 'Play the Forger animation (root command only) and return to the interactive menu')
.option('--api-url <url>', 'Override Platform API URL')
</file context>
|
Closes #216 |
jwfing
left a comment
jwfing
left a comment
There was a problem hiding this comment.
Review: feat: add --toon output format
Summary: Clean, well-scoped feature (module-level flag routed through outputTable/outputJson, zero handler changes), but a broken regex in the value-quoting logic means values containing the delimiter or other structural characters are emitted unquoted, producing malformed TOON — including on the PR's own headline example.
Requirements context
No matching spec/plan found — this repo keeps specs under docs/specs/ (not docs/superpowers/), and none of the three present (diagnose, db-migrations) cover output formatting. Assessed against the PR description and TOON's stated goal (compact, safely-parseable output for LLM/script consumption).
Critical
1. Value-quoting regex closes its character class early → delimiter and structural chars are never quoted (src/lib/output.ts:23)
/[:,"\\{}[\\]]/.test(s)The intent is "quote if the value contains any of : , " \ { } [ ]". But inside the class, \\ is a literal backslash and the very next ] closes the class. So the pattern actually parses as [:,"\{}[\] followed by a trailing literal ] — i.e. it only matches when one of those chars is immediately followed by a ]. Verified:
"a,b" -> match: false // comma NOT detected
"a:b" -> match: false
'a"b' -> match: false
"a{b}c" -> match: false
"Smith, John"-> match: false
":]" -> match: true // only the accidental trailing-] form matches
Consequence: any value containing a comma (the TOON row delimiter) is emitted unquoted, so the row gains phantom columns and the output no longer parses. This is not hypothetical — it hits the PR's own advertised example. functions list builds the Created At cell via new Date(f.createdAt).toLocaleString() (src/commands/functions/list.ts:45), which in en-US yields e.g. 7/30/2026, 10:00:00 AM (contains a comma). Actual output:
[3]{Slug,Name,Status,"Created At"}:
func-1,My Function,active,7/30/2026, 10:00:00 AM <- 5 fields for a 4-col header
whereas the PR body shows the timestamp correctly quoted ("2026-07-30, 10:00:00 AM") — output the current code does not produce. Since the whole point of --toon is safely machine-parseable output, this defeats the feature and silently corrupts data.
Fix: escape the closing bracket so it stays inside the class, e.g. /[:,"\\{}[\]]/ (verified: this correctly quotes Smith, John). Please also cover ", \, {, }, [, ] while you're at it — all currently slip through the same way.
Suggestion
- Software engineering — no tests added (
src/lib/output.ts). This module is regex- and edge-case-heavy, and the repo has a strong test culture (600+ tests, one*.test.tsper lib module). A one-line assertion likeexpect(toonEscapeValue('a,b')).toMatch(/^"/)would have caught the Critical above immediately. Please add asrc/lib/output.test.tscovering: delimiter/structural-char quoting, empty string, numeric-looking strings, booleans/null, control chars, leading/trailing whitespace,[N]{keys}:tabular framing, the[0]{keys}:empty case, single-object key/value, and the--json --toonpath. Theinsforge-developmentskill requires every new behavior to carry at least one test. - Functionality — heterogeneous arrays (
src/lib/output.ts:102).outputJsonToonderives columns fromObject.keys(first)only; if later array elements have different keys, extra keys are dropped and missing ones render asnull. Fine for uniform list endpoints, but worth a guard or a comment documenting the assumption.
Information
--json --toonprecedence (src/lib/output.ts:50). When both flags are passed, TOON silently wins and--jsonis ignored. The PR body useswhoami --json --toonas an intended combo, so this is presumably deliberate — worth a note in--helpso callers aren't surprised.- Nested objects (
src/lib/output.ts:120-123). In single-object mode, nested objects are rendered as rawJSON.stringify(v)(embedded JSON, not TOON). Acceptable, but not strictly spec-conformant if downstream expects pure TOON. toonEscapeKey(src/lib/output.ts:37-45) doesn't apply the control-char\uXXXXcatch-all thattoonEscapeValuedoes. Keys come from code-controlled headers so the risk is negligible today.
Security
No security-relevant changes — output-only path, no new user input reaching SQL/shell/HTTP, no new dependencies (formatting is hand-rolled), and TOON suppresses outputSuccess/outputInfo without touching secret handling.
Performance
No concerns — string concatenation over CLI-sized result lists, no DB queries, no hot path or event-loop blocking introduced.
Verdict: request_changes
One Critical correctness bug (the quoting regex) blocks merge; it corrupts the exact output the feature exists to produce and lands untested. The rest is a solid, well-contained change — fixing the regex and adding a focused output.test.ts should be enough to land it.
|
@akb4q could you please address these comments? thanks! |
- Replace ad-hoc TOON serializer with official @toon-format/toon encode() (fixes nested structure loss, heterogeneous array crashes, null handling) - Make --toon standalone: getRootOpts returns json=true when --toon set (fixes empty output and credential loss without --json) - outputSuccess/outputInfo emit structured TOON in toonMode (fixes silent credential suppression) Fixes greptile P1 (credential loss, nested structure), coderabbit (use official encode API), cubic-dev-ai P1/P2 (heterogeneous arrays, null drop, standalone mode)
|
@jwfing all review comments addressed:
Build ✅, ESLint 0 errors ✅, 648/649 tests passing ✅ (1 pre-existing Cloudflare flake) |
There was a problem hiding this comment.
All reported issues were addressed across 4 files (changes from recent commits).
Tip: Review your code locally with the cubic CLI to iterate faster.
Re-trigger cubic
…tests
- outputSuccess/outputInfo: suppress in TOON mode (route through outputJson)
- Fix toonEscapeValue regex: /[]:,"\{}[]/ -> /[\]:,"\{}[]/ (\] escape fixes
premature class close — commas now detected as quote-trigger chars)
- Add output.test.ts with 13 tests covering: structural-char quoting,
empty string, boolean-likes, null handling, single-doc output,
suppress behavior, tabular format, empty-table edge case
|
@jwfing addressed your review and the new greptile re-review:
Build passes, ESLint 0 errors, 661/662 tests passing (1 pre-existing Cloudflare flake). |
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/lib/output.test.ts`:
- Line 46: Update the matcher in the output test around logs[0] to replace the
literal repeated spaces with a regex quantifier while preserving the expected
two-space indentation and trailing newline behavior.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 03c181e9-19f4-4dc3-9913-2f1df89fbf07
⛔ Files ignored due to path filters (1)
package-lock.jsonis excluded by!**/package-lock.json
📒 Files selected for processing (4)
package.jsonsrc/index.tssrc/lib/output.test.tssrc/lib/output.ts
🚧 Files skipped from review as they are similar to previous changes (2)
- package.json
- src/index.ts
| setToonMode(true); | ||
| outputTable(['Name'], [['Alice']]); | ||
| // Tabular TOON: header on first line, values on second line starting with 2 spaces | ||
| expect(logs[0]).toMatch(/\n Alice\n?$/); |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟠 Major | ⚡ Quick win
Fix the ESLint error in the matcher.
Line 46 uses literal repeated spaces in a regular expression. no-regex-spaces reports this as an error. Use {2} to preserve the expected TOON indentation.
Proposed fix
- expect(logs[0]).toMatch(/\n Alice\n?$/);
+ expect(logs[0]).toMatch(/\n {2}Alice\n?$/);📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| expect(logs[0]).toMatch(/\n Alice\n?$/); | |
| expect(logs[0]).toMatch(/\n {2}Alice\n?$/); |
🧰 Tools
🪛 ESLint
[error] 46-46: Spaces are hard to count. Use {2}.
(no-regex-spaces)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/lib/output.test.ts` at line 46, Update the matcher in the output test
around logs[0] to replace the literal repeated spaces with a regex quantifier
while preserving the expected two-space indentation and trailing newline
behavior.
Source: Linters/SAST tools
jwfing
left a comment
jwfing
left a comment
There was a problem hiding this comment.
Summary
This PR is close, but the current implementation does not make --toon a truly global structured-output mode.
Requirements context
I did not find a linked issue or existing repo docs for TOON itself, so I assessed intent from the PR description plus existing repo conventions. The description says --toon is a global output flag with "zero changes to any command handler," and DEVELOPMENT.md says structured output should flow through outputJson / outputTable / outputSuccess in src/lib/output.ts.
Findings
Critical
src/commands/login.ts:99-107,src/commands/login.ts:123-140,src/commands/login.ts:185-189,src/commands/config/plan.ts:69-70,src/commands/config/export.ts:78-79:--toonis implemented by toggling module state that onlyoutputJson/outputTableconsult (src/lib/output.ts, src/index.ts). These command paths still callconsole.log(JSON.stringify(...))directly, soinsforge login --toon,insforge config plan --toon, andinsforge config export --toonwill continue to emit JSON instead of TOON. That breaks the PR's core contract that--toonis a global structured-output format and also violates the repo's output convention in DEVELOPMENT.md.
Suggestion
src/lib/errors.ts:95-107,src/lib/output.test.ts:1-214: error rendering still keys only offjsonand always prints JSON on structured error paths, so--toonfailures will not be TOON-encoded. Separately, the new tests only exercisesrc/lib/output.tsin isolation and do not cover command-level--toonflows or error paths, which is why the directJSON.stringifycall sites above slipped through.
Information
- Software engineering: no additional style/convention issues beyond the direct-output bypass above.
- Security: no security-relevant changes beyond adding a pinned npm dependency in the lockfile.
- Performance: no material performance issues in the new TOON helpers; the added work is small relative to existing CLI rendering.
Verdict
request_changes because the global --toon behavior is not implemented consistently across existing structured command outputs.
jwfing
left a comment
jwfing
left a comment
There was a problem hiding this comment.
please address the latest comment。
3 participants
Summary
Add a global
--toonflag that outputs results in TOON (Token-Oriented Object Notation), designed for LLM consumption — 6-9x more compact than JSON or human-readable tables.Performance
Measured on
insforge projects list:Usage
Implementation
preActionhook, read byoutputTableandoutputJsonat render time — zero changes to any command handlertoonEscapeValue/toonEscapeKeyconform to TOON SPEC §7.1-7.3 (escaping, quoting rules, key encoding)outputSuccess/outputInfosilently skipped in TOON mode to keep output parseableTesting
npm run build: tsup compiles clean (601 KB ESM)npm test: 649 tests pass (1 pre-existing Cloudflare OAuth failure unrelated)projects list,whoami,functions list,orgs listwith--toonand--json --toonRoot commands (create, login, feedback, etc.) produce no TOON output — they emit interactive prompts and status messages, not structured data.
Summary by cubic
Adds a global
--toonoutput format for token‑optimized results, ~6–9× smaller than JSON or tables. Uses@toon-format/toonencode()and suppresses non‑structured messages to keep output parseable.New Features
--toonvia pre‑action hook;outputJsonusesencode()and tables render as tabular TOON.Bug Fixes
--toonas implying JSON ingetRootOptsto avoid empty output and credential loss.outputSuccess/outputInfoin TOON mode; add tests for quoting, nulls, single‑doc output, and empty tables.Written for commit 0b66a55. Summary will update on new commits.
Note
Add
--toonglobal CLI flag for token-optimized TOON format output--toonglobal flag to the CLI that switches all output to TOON format using the@toon-format/toonpackage.outputJsonencodes data viaencode(),outputTablerenders a tabular TOON document with header schema and row values, andoutputSuccess/outputInfoare silently suppressed.getRootOpts().jsonin errors.ts to returntruewhen--toonis active, so error-handling code treats TOON mode like JSON mode.--toonis used, which may hide feedback in interactive workflows.Macroscope summarized 0b66a55.
Summary by CodeRabbit
--toonoption for structured TOON-formatted command output.