docs(runner): make the guide tell the truth, and make drift fail a test (DEV-2203) - #188
Open
danielzytohoc wants to merge 2 commits into
Open
docs(runner): make the guide tell the truth, and make drift fail a test (DEV-2203)#188danielzytohoc wants to merge 2 commits into
danielzytohoc wants to merge 2 commits into
Conversation
…st (DEV-2203) A parity audit read the five guide tracks against the product they describe and found fourteen places where they disagree. The guide loses every one of those arguments, so the text moves; and four new parity tests in guide-tracks.test.mjs read the product's own files so the next drift fails CI instead of a reader. Stale status: - everyone: the 'landing shortly' block for updating by asking is gone — PATCH /api/mcp/demos/:id is merged, so asking works now, with the real boundary stated (MCP-created demos only; browser demos answer 403 and point at /edit). Dead URLs: - ?docs= examples pointed at guides/columns/column-adding and guides/rows/row-sorting, which exist in no bucket; every printed URL now names rows-sorting/react/exampleSortingDemo.tsx, which is in 18.0. - the versioned ?docs= example used v=17.1.0, a version with no docs bucket; it pins 18.0.0. - /all-demos?owner= used a hyphenated slug; the slug is the email local part, dots included. Over-promises: - the budget prose claimed saving and forking survive the ceiling; a Save or Fork boots a build container and the top tiers refuse it, so the guide now promises only what R2 serves — existing links and embeds. - 'imports and file drops with no Handsontable are refused' — only imports are guarded; the drop path has no such check. - 'node_modules, build output and lockfiles are refused, not dropped' — true on the MCP path only; the file drop skips them quietly and stops at 50 files, saying where it stopped. - Download sat in the signed-in Files list; it has never been auth-gated. Naming: - the guide handed /d/<id>/ out as 'the client link', but both product surfaces (Share's 'Public client link', My demos' Copy link) hand out /share/<id>; the guide now describes what the product does, with /d/<id>/ as the bare static page the playground displays. - 'This demo is not available' is a string nowhere in the product; the real one is 'This demo is unavailable.' Precision: - the MCP path requires a description (400 without one); said so. - bare integers: 15-19 are majors, 20-999 are refused, 1000 and up are pkg.pr.new refs — 'below 1000 is a version' was two-thirds wrong. - the 'Container (tier 2)' list named four tier-1 starters; the section speaks in engines now, and admits React (Vite, JS) is a container too. - /d/<id>/ 'cannot be framed at all, on any site' — frame-ancestors is 'self', which is how /share displays it; now 'on any other site'. The four new tests: every printed ?docs= path must exist in the 18.0 manifest; every stated number must equal the product constant it came from (MCP caps, description ceiling, major range, PR-ref floor, payload TTL); every ?example= key must be in the catalog; and every starter the container list names must have engine 'container' in catalog.json, with catalog-tier vocabulary banned from that section outright.
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit b07cbc6. Configure here.
…Bugbot #188) Master rewrote 'Changing a demo, also by asking' for the shipped update_demo tool — its version wins, with the one fact it lacked grafted in: updating only works on demos the MCP created (a browser-built demo answers with the /edit/<id> refusal, index.ts ~777), which is a separate boundary from ownership. The numbers parity test now also pins MAX_DROP_FILES from dropFiles.ts: the guide's 'stops at 50 files' named a constant the test never read, so the drop claim could drift while the MCP caps stayed green.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

A parity audit (DEV-2203) read the five guide tracks in
runner/docs/guide/against the product they describe. Fourteen claims lost the argument. This PR fixes the text and adds four parity tests torunner/pipeline/guide-tracks.test.mjsso the next drift fails CI instead of a reader. No product code changes.The fourteen fixes, grouped
Stale status
everyone.md— deleted the "Status: landing shortly" block and its two interim workarounds:PATCH /api/mcp/demos/:idis merged (workers/api/src/index.ts), so changing a demo by asking works now.everyone.md— added the real boundary: only MCP-created demos can be updated this way; browser-built demos answer 403 with "this demo was not created through the MCP; edit it at /edit/ in the browser".Dead URLs
3. All printed
?docs=URLs now nameguides/rows/rows-sorting/react/exampleSortingDemo.tsx, which exists in the 18.0 bucket —column-addingis not in 18.0 androws/row-sortingexists in no bucket (the guide isrows-sorting). The versioned example pins&v=18.0.0; there is no 17.x docs bucket.4.
support.md—/all-demos?owner=marek-martuszewski→owner=marek.martuszewski; the slug is the email local part (demoOwners.ts), dots included.Over-promises
5.
overview.md+developers.md— the budget prose claimed "saving or forking still works" while paused.budgetGatealso gatesPOST /api/demosand the rebuild, so a Save/Fork is refused at the top tiers. The guide now promises only the static reads.6.
overview.md— "Imports and file drops with no Handsontable in them are refused": only imports are guarded (import-url.ts);dropFiles.tshas no such check. Now imports only.7.
support.md— Download sat in the "Signed in, Files also lets you" list; it has never been auth-gated (TopBar.tsxsays so explicitly). Moved out of the signed-in list.14.
developers.md— "node_modules, build output and lockfiles are refused, not dropped" was claimed for every path; it is true for the MCP only. The file drop skips them quietly and stops at 50 files (with a "Stopped at 50 files" report). Split per path. (Note: the audit called the 50-file cut silent;dropFiles.tsreports it —truncatedfeedsrejectionMessage()— so the guide says "stops … saying where it stopped".)Naming
8.
support.md+overview.md+devrel.md— the guide handed out/d/<id>/as "the client link", but both product surfaces (Share's Public client link field, My demos' Copy link) hand out/share/<id>. The guide now documents actual behavior:/share/<id>is the public client link (read-only playground);/d/<id>/is the bare static page it displays.10.
everyone.md— the quoted error "This demo is not available" exists nowhere; replaced with the real string "This demo is unavailable." (App.tsx; the revoked page says "This demo has been revoked",share.ts).Precision
9.
everyone.md— the MCP path requires a description (400 without one,index.ts), unlike the editor. Said so.11.
devrel.md—/d/<id>/"cannot be framed at all, on any site": CSP isframe-ancestors 'self', which is exactly how/shareand full mode display it. Now "on any other site".12.
developers.md— "Integers below 1000 are read as a version": 15–19 are majors, 20–999 are refused outright (MIN_BARE_NUMERIC_PKG_PR_NEW_REF), ≥1000 is a build ref. Made precise.13.
developers.md— "Container (tier 2) — … MUI, Ant Design, Fluent UI, Base Web": those four aretier: 1withengine: "container"incatalog.json. The section speaks in engines now (asoverview.mddoes) and admits React (Vite, JS) also runs in a container.The four parity tests
?docs=path from the five docs, asserts it is adocsPathinapps/authoring/public/docs-examples/18.0/manifest.json.MAX_MCP_FILES/MAX_MCP_BYTES(mcp-create.ts),MAX_DESCRIPTION(demo-info.ts),DEFAULT_MIN_MAJOR/DEFAULT_MAX_MAJOR(built runtime), and source-greps the unexportedMIN_BARE_NUMERIC_PKG_PR_NEW_REFandPAYLOAD_TTL_SECONDS; asserts the markdown states each (50, 256 KB, 4,000, 15–19, 1000, 24 hours).?example=<key>(and the URL table's backticked alternates) is a framework key inrunner/catalog.json.engine === "container"/"sandpack"; catalog-tier vocabulary is banned from that section (the assertion that would have caught fix 13).Spot-checked: re-breaking one
?docs=URL fails test 1; restoring the "(tier 2)" wording fails test 4.Needs a product decision (flagging for Dan)
The client-link naming tension is documented, not resolved: the product hands out
/share/<id>under the label "Public client link" (ShareLinks.tsx) and from My demos' Copy link (MyDemos.tsx), while the docs vocabulary used to reserve "client link" for/d/<id>/. The guide now documents current behavior. If the intent was for clients to receive/d/<id>/links, that is a product change toApp.tsx/MyDemos.tsx— not made here.Test results
pnpm test— 364 pass (incl. the 4 new parity tests)pnpm e2e e2e/guide.spec.ts— 10 passNote
Low Risk
Documentation and test-only changes; no runtime or API behavior is modified in this PR.
Overview
Runner guide tracks (
developers,devrel,everyone,overview,support) are rewritten so URLs, limits, starter engines, budget behavior, and sharing vocabulary match the live product—without changing application code.Docs fixes include swapping printed
?docs=examples toguides/rows/rows-sorting/react/exampleSortingDemo.tsx, treating/share/<id>as the public client link vs/d/<id>/as the bare static page, correcting budget-pause text (Save/Fork blocked at top tiers; static reads still work), splitting MCP vs file-drop handling ofnode_modules/lockfiles, noting MCP-required descriptions, fixing owner filter slug (marek.martuszewski), clarifying bare version integers (15–19majors,20–999refused, PR refs from1000), and describing container vs in-browser engines (including React Vite JS in a container).CI adds four parity tests in
guide-tracks.test.mjs: every guide?docs=path exists in the 18.0 manifest, numeric limits matchmcp-create,demo-info,dropFiles, andversionconstants,?example=keys matchcatalog.json, anddevelopers.mdcontainer/in-browser lists match catalogenginevalues.Reviewed by Cursor Bugbot for commit 7c1712e. Bugbot is set up for automated code reviews on this repo. Configure here.