From 2ab0240de72197e04ef3b450aafb413b9f2d6a62 Mon Sep 17 00:00:00 2001 From: Guilherme Souza Date: Fri, 19 Jun 2026 10:11:49 -0300 Subject: [PATCH 1/9] docs: add design spec for TypeScript parser typedoc refactor --- .../2026-06-19-ts-parser-typedoc-design.md | 131 ++++++++++++++++++ 1 file changed, 131 insertions(+) create mode 100644 docs/specs/2026-06-19-ts-parser-typedoc-design.md diff --git a/docs/specs/2026-06-19-ts-parser-typedoc-design.md b/docs/specs/2026-06-19-ts-parser-typedoc-design.md new file mode 100644 index 0000000..497fea7 --- /dev/null +++ b/docs/specs/2026-06-19-ts-parser-typedoc-design.md @@ -0,0 +1,131 @@ +# TypeScript Public API Parser: typedoc Refactor + +**Date:** 2026-06-19 +**Status:** Approved + +## Problem + +`ts-parser.ts` walks individual TypeScript source files using the TypeScript compiler API in syntactic-only mode. It correctly handles top-level `class`, `function`, and `variable` declarations but misses: + +- `interface` and `type` alias declarations +- `enum` declarations and members +- Re-exports (`export * from`, `export { foo } from`) +- Namespace merging and declaration merging +- Symbols only accessible via the package's `exports` field + +This creates false negatives: new public symbols added to the supabase-js SDK may not be caught by the compliance check because the parser cannot see them. + +There is also an architectural inconsistency. PR #35 established the pattern of using an external, well-maintained language tool (dartdoc_json) to generate structured JSON, then normalizing that output into `ParseResult`. The TypeScript path should follow the same shape. + +## Solution + +Replace `ts-parser.ts` and `parse-ts.ts` with a thin normalizer (`normalize-typedoc.ts`) over [TypeDoc](https://typedoc.org/) JSON output. TypeDoc runs the full TypeScript compiler internally and handles all of the cases the current parser misses. The `check-api-symbols` interface and everything downstream are unchanged. + +## End-to-End Flow + +``` +supabase-js PR + → validate-sdk-compliance.yml + [ts-only] npm ci (in SDK repo) + [ts-only] npx typedoc@0.27 --json pr-raw.json --excludePrivate --excludeProtected + [ts-only] git checkout base; npm ci; npx typedoc@0.27 --json base-raw.json … + → normalize-typedoc pr-raw.json → pr-symbols.json + → normalize-typedoc base-raw.json → base-symbols.json + → check-api-symbols pr-symbols.json base-symbols.json sdk-compliance.yaml +``` + +## Files Changed + +### Added +- `scripts/capability-matrix/src/normalize-typedoc.ts` — core normalizer +- `scripts/capability-matrix/src/normalize-typedoc-cli.ts` — CLI entry point +- `scripts/capability-matrix/test/normalize-typedoc.test.ts` — ~15 fixture-based unit tests +- `scripts/capability-matrix/test/fixtures/typedoc-sample.json` — real TypeDoc 0.27 output + +### Deleted +- `scripts/capability-matrix/src/ts-parser.ts` +- `scripts/capability-matrix/src/parse-ts.ts` +- `scripts/capability-matrix/test/ts-parser.test.ts` +- `scripts/capability-matrix/test/fixtures/ts-sample/` + +### Modified +- `.github/workflows/validate-sdk-compliance.yml` — replace `parse-ts` steps with TypeDoc steps; add `entrypoint` input +- `scripts/capability-matrix/package.json` — `normalize-typedoc` script replaces `parse-ts` + +## Normalizer Design + +TypeDoc JSON is a tree of "reflections", each with a numeric `kind`. The normalizer walks two levels: top-level declarations and their members for class-like types. + +### Tree traversal + +With `--entryPointStrategy resolve`, TypeDoc wraps each entrypoint's declarations inside a `Module` reflection (kind 2) rather than placing them directly under the project root. The normalizer flattens one level: it walks `project.children`, and for any child with kind `Module` (2) or `Namespace` (4) it recurses into that child's `children` to find declarations. Namespaces are not emitted as symbols themselves — only their members are. + +### Top-level kind mapping + +| TypeDoc kind | Numeric value | Emitted `ParsedSymbol.kind` | Notes | +|---|---|---|---| +| Class | 128 | `"class"` | walk members | +| Interface | 256 | `"class"` | treated same as class | +| Enum | 8 | `"class"` | walk members | +| Function | 64 | `"function"` | | +| Variable | 32 | `"variable"` | | +| TypeAlias | 2097152 | `"variable"` | exported types worth tracking | +| Reference | 4194304 | skip | re-export pointer; canonical declaration already emitted | + +### Member kind mapping (inside Class / Interface / Enum) + +| TypeDoc kind | Numeric value | Emitted `ParsedSymbol.kind` | +|---|---|---| +| Method | 2048 | `"method"` with `ClassName.name` | +| Property | 1024 | `"property"` with `ClassName.name` | +| Accessor | 262144 | `"method"` (getter/setter) | +| Constructor | 512 | skip | +| EnumMember | 16 | `"property"` with `EnumName.MemberName` | + +### Privacy + +TypeDoc is invoked with `--excludePrivate --excludeProtected`, so private and protected members are stripped before the JSON is written. The normalizer also defensively skips any member where `flags.isPrivate` or `flags.isProtected` is `true`, in case the tool is ever called without those flags. + +### File path + +`ParsedSymbol.file` is populated from `sources[0].fileName` on each reflection (e.g. `src/auth/client.ts`). + +### Normalizer public interface + +```typescript +export function normalize(json: unknown): ParseResult +``` + +A single function. The CLI wrapper reads the input file, calls `normalize`, and writes the output file. + +## CI Workflow Changes + +New `entrypoint` input on `validate-sdk-compliance.yml` (default: `src/index.ts`). SDK repos that have a non-standard entrypoint pass this explicitly; most do not need to change. + +TypeDoc is pinned at `0.27` and invoked via `npx --yes typedoc@0.27` — no changes needed to the SDK's `package.json`. + +The `language: javascript` input value is unchanged; SDK repos that already call this workflow need no modifications. + +## Testing + +The fixture (`typedoc-sample.json`) is generated from a minimal TypeScript project that exercises all mapped cases. It is committed to the repo so tests are hermetic and do not require a TypeScript build at test time. + +Test coverage (~15 cases): + +| Case | Assertion | +|---|---| +| Class | emits `"class"` kind | +| Class methods | `ClassName.method` as `"method"` | +| Class property | `ClassName.prop` as `"property"` | +| Accessor | getter/setter as `"method"` | +| Constructor | not emitted | +| Interface | emitted as `"class"` | +| Interface members | same rules as class members | +| Enum | emitted as `"class"` | +| EnumMember | `EnumName.Member` as `"property"` | +| Exported function | `"function"` kind | +| Exported variable | `"variable"` kind | +| TypeAlias | `"variable"` kind | +| Reference (re-export) | not emitted | +| File path | taken from `sources[0].fileName` | +| Defensive privacy | member with `flags.isPrivate: true` not emitted | From 1a3f768dfadd1ac433a12cc1faee65f75cfb9024 Mon Sep 17 00:00:00 2001 From: Guilherme Souza Date: Fri, 19 Jun 2026 11:03:20 -0300 Subject: [PATCH 2/9] docs: add implementation plan for TypeScript parser typedoc refactor --- docs/plans/2026-06-19-ts-parser-typedoc.md | 754 +++++++++++++++++++++ 1 file changed, 754 insertions(+) create mode 100644 docs/plans/2026-06-19-ts-parser-typedoc.md diff --git a/docs/plans/2026-06-19-ts-parser-typedoc.md b/docs/plans/2026-06-19-ts-parser-typedoc.md new file mode 100644 index 0000000..d5382ab --- /dev/null +++ b/docs/plans/2026-06-19-ts-parser-typedoc.md @@ -0,0 +1,754 @@ +# TypeScript Parser TypeDoc Refactor Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Replace `ts-parser.ts` (hand-rolled TypeScript AST walker) with a thin normalizer over TypeDoc 0.27 JSON output, following the same "external tool → normalizer → ParseResult" architecture as the Dart parser added in PR #35. + +**Architecture:** TypeDoc 0.27 runs in CI inside the SDK repo (after `npm ci`), producing a JSON file of all public declarations. `normalize-typedoc.ts` maps TypeDoc's reflection kinds to the existing `ParseResult` shape. `check-api-symbols` and everything downstream are untouched. `ParsedSymbol`/`ParseResult` type definitions move from `ts-parser.ts` to `normalize-typedoc.ts`; `swift-parser.ts` re-imports from there. + +**Tech Stack:** TypeScript 5, TypeDoc 0.27, Vitest 4, Node 22, `tsx` for dev scripts + +## Global Constraints + +- TypeDoc pinned at `0.27` — invoked as `npx --yes typedoc@0.27` +- `ParsedSymbol.kind` stays `"class" | "method" | "property" | "function" | "variable"` — no new values +- `check-api-symbols.ts` and `api-check.ts` are not modified +- All existing tests must pass after cleanup +- Conventional commits: `feat:`, `refactor:`, `test:`, `ci:` +- All scripts in `package.json` use `tsx` prefix (matches existing pattern) +- Working directory for capability-matrix commands: `scripts/capability-matrix/` + +--- + +### Task 1: Generate typedoc-sample.json fixture + +**Files:** +- Create: `scripts/capability-matrix/test/fixtures/typedoc-sample.json` + +**Interfaces:** +- Produces: real TypeDoc 0.27 JSON consumed by fixture-based tests in Task 2 + +- [ ] **Step 1: Create a minimal TypeScript project in a temp directory** + +```bash +mkdir -p /tmp/typedoc-fixture/src +``` + +Write `/tmp/typedoc-fixture/src/index.ts`: +```typescript +export class AuthClient { + constructor(private url: string) {} + signUp(email: string): void {} + signIn(): void {} + get session(): string { return ""; } + private _cache = new Map(); +} + +export interface Session { + user: string; + expires: Date; +} + +export enum UserRole { + Admin = "admin", + User = "user", +} + +export type AuthResponse = { data: unknown; error: Error | null }; + +export function createClient(url: string): AuthClient { + return new AuthClient(url); +} + +export const VERSION = "1.0.0"; + +export { AuthClient as Client }; +``` + +Write `/tmp/typedoc-fixture/tsconfig.json`: +```json +{ + "compilerOptions": { + "target": "ES2020", + "module": "NodeNext", + "moduleResolution": "NodeNext", + "strict": true + }, + "include": ["src"] +} +``` + +Write `/tmp/typedoc-fixture/package.json`: +```json +{ "name": "fixture", "version": "1.0.0", "type": "module" } +``` + +- [ ] **Step 2: Install TypeDoc and generate JSON** + +```bash +cd /tmp/typedoc-fixture +npm install --save-dev typedoc@0.27 +npx typedoc --json api.json --excludePrivate --excludeProtected src/index.ts +``` + +Expected: `api.json` is created. TypeDoc may emit warnings — these are fine as long as the file is produced. + +- [ ] **Step 3: Copy the fixture into the repo** + +```bash +cp /tmp/typedoc-fixture/api.json \ + /scripts/capability-matrix/test/fixtures/typedoc-sample.json +``` + +Replace `` with the actual absolute path to the repo. + +- [ ] **Step 4: Sanity-check the fixture** + +```bash +cd scripts/capability-matrix +node --input-type=module <<'EOF' +import { readFileSync } from "fs"; +const f = JSON.parse(readFileSync("test/fixtures/typedoc-sample.json", "utf8")); +const walk = (children) => children.flatMap(c => c.kind === 2 ? walk(c.children ?? []) : [c]); +console.log(walk(f.children ?? []).map(c => `${c.kind} ${c.name}`)); +EOF +``` + +Expected output includes lines like `128 AuthClient`, `256 Session`, `8 UserRole`, `64 createClient`, `32 VERSION`, `2097152 AuthResponse` and a Reference line for `Client`. + +- [ ] **Step 5: Commit** + +```bash +git add scripts/capability-matrix/test/fixtures/typedoc-sample.json +git commit -m "test: add real typedoc-sample.json fixture for normalizer tests" +``` + +--- + +### Task 2: TDD — write tests, implement normalize-typedoc.ts + +**Files:** +- Create: `scripts/capability-matrix/src/normalize-typedoc.ts` +- Create: `scripts/capability-matrix/test/normalize-typedoc.test.ts` +- Modify: `scripts/capability-matrix/src/swift-parser.ts` (update import path only) + +**Interfaces:** +- Produces: `normalize(json: unknown): ParseResult` exported from `normalize-typedoc.ts` +- Produces: `ParsedSymbol`, `ParseResult` types exported from `normalize-typedoc.ts` (replacing `ts-parser.ts` as the canonical source) + +- [ ] **Step 1: Write the failing tests** + +Create `scripts/capability-matrix/test/normalize-typedoc.test.ts`: + +```typescript +import { describe, it, expect } from "vitest"; +import { join, dirname } from "node:path"; +import { fileURLToPath } from "node:url"; +import { readFileSync } from "node:fs"; +import { normalize } from "../src/normalize-typedoc.js"; + +const __dirname = dirname(fileURLToPath(import.meta.url)); + +// ── Inline JSON helpers ────────────────────────────────────────────────────── +// Each builds the minimal TypeDoc shape needed for a specific case. + +function project(...children: object[]) { + return { kind: 1, name: "test", children }; +} +function mod(name: string, ...children: object[]) { + return { kind: 2, name, flags: {}, children }; +} +function cls(name: string, file: string, ...members: object[]) { + return { kind: 128, name, flags: {}, sources: [{ fileName: file }], children: members }; +} +function iface(name: string, file: string, ...members: object[]) { + return { kind: 256, name, flags: {}, sources: [{ fileName: file }], children: members }; +} +function enumDecl(name: string, file: string, ...members: object[]) { + return { kind: 8, name, flags: {}, sources: [{ fileName: file }], children: members }; +} +function method(name: string, file: string) { + return { kind: 2048, name, flags: {}, sources: [{ fileName: file }] }; +} +function prop(name: string, file: string) { + return { kind: 1024, name, flags: {}, sources: [{ fileName: file }] }; +} +function accessor(name: string, file: string) { + return { kind: 262144, name, flags: {}, sources: [{ fileName: file }] }; +} +function ctor(file: string) { + return { kind: 512, name: "constructor", flags: {}, sources: [{ fileName: file }] }; +} +function enumMember(name: string, file: string) { + return { kind: 16, name, flags: {}, sources: [{ fileName: file }] }; +} +function fn(name: string, file: string) { + return { kind: 64, name, flags: {}, sources: [{ fileName: file }] }; +} +function variable(name: string, file: string) { + return { kind: 32, name, flags: {}, sources: [{ fileName: file }] }; +} +function typeAlias(name: string, file: string) { + return { kind: 2097152, name, flags: {}, sources: [{ fileName: file }] }; +} +function ref(name: string) { + return { kind: 4194304, name, flags: {} }; +} +function privateFlag(base: object): object { + return { ...base, flags: { isPrivate: true } }; +} +function protectedFlag(base: object): object { + return { ...base, flags: { isProtected: true } }; +} + +// ── Unit tests (inline JSON) ───────────────────────────────────────────────── + +describe("normalize — class", () => { + it("emits class symbol", () => { + const result = normalize(project(cls("AuthClient", "src/auth.ts"))); + expect(result.symbols).toContainEqual({ name: "AuthClient", kind: "class", file: "src/auth.ts" }); + }); + + it("emits class method", () => { + const result = normalize(project(cls("AuthClient", "src/auth.ts", method("signUp", "src/auth.ts")))); + expect(result.symbols).toContainEqual({ name: "AuthClient.signUp", kind: "method", file: "src/auth.ts" }); + }); + + it("emits class property", () => { + const result = normalize(project(cls("AuthClient", "src/auth.ts", prop("session", "src/auth.ts")))); + expect(result.symbols).toContainEqual({ name: "AuthClient.session", kind: "property", file: "src/auth.ts" }); + }); + + it("emits accessor as method kind", () => { + const result = normalize(project(cls("AuthClient", "src/auth.ts", accessor("token", "src/auth.ts")))); + expect(result.symbols).toContainEqual({ name: "AuthClient.token", kind: "method", file: "src/auth.ts" }); + }); + + it("skips constructor", () => { + const result = normalize(project(cls("Foo", "src/foo.ts", ctor("src/foo.ts"), method("bar", "src/foo.ts")))); + const names = result.symbols.map(s => s.name); + expect(names).not.toContain("Foo.constructor"); + expect(names).toContain("Foo.bar"); + }); +}); + +describe("normalize — interface", () => { + it("emits interface as class kind", () => { + const result = normalize(project(iface("Session", "src/session.ts"))); + expect(result.symbols).toContainEqual({ name: "Session", kind: "class", file: "src/session.ts" }); + }); + + it("emits interface members as property", () => { + const result = normalize(project(iface("Session", "src/session.ts", prop("user", "src/session.ts")))); + expect(result.symbols).toContainEqual({ name: "Session.user", kind: "property", file: "src/session.ts" }); + }); +}); + +describe("normalize — enum", () => { + it("emits enum as class kind", () => { + const result = normalize(project(enumDecl("UserRole", "src/role.ts"))); + expect(result.symbols).toContainEqual({ name: "UserRole", kind: "class", file: "src/role.ts" }); + }); + + it("emits enum member as property kind", () => { + const result = normalize(project(enumDecl("UserRole", "src/role.ts", enumMember("Admin", "src/role.ts")))); + expect(result.symbols).toContainEqual({ name: "UserRole.Admin", kind: "property", file: "src/role.ts" }); + }); +}); + +describe("normalize — top-level declarations", () => { + it("emits exported function", () => { + const result = normalize(project(fn("createClient", "src/index.ts"))); + expect(result.symbols).toContainEqual({ name: "createClient", kind: "function", file: "src/index.ts" }); + }); + + it("emits exported variable", () => { + const result = normalize(project(variable("VERSION", "src/index.ts"))); + expect(result.symbols).toContainEqual({ name: "VERSION", kind: "variable", file: "src/index.ts" }); + }); + + it("emits type alias as variable kind", () => { + const result = normalize(project(typeAlias("AuthResponse", "src/types.ts"))); + expect(result.symbols).toContainEqual({ name: "AuthResponse", kind: "variable", file: "src/types.ts" }); + }); + + it("skips Reference kind", () => { + const result = normalize(project(cls("AuthClient", "src/auth.ts"), ref("Client"))); + const names = result.symbols.map(s => s.name); + expect(names).not.toContain("Client"); + expect(names).toContain("AuthClient"); + }); +}); + +describe("normalize — traversal", () => { + it("walks into Module wrapper (kind 2)", () => { + const result = normalize(project(mod("src/auth", cls("AuthClient", "src/auth.ts")))); + expect(result.symbols).toContainEqual({ name: "AuthClient", kind: "class", file: "src/auth.ts" }); + }); + + it("captures file path from sources[0].fileName", () => { + const result = normalize(project(fn("foo", "packages/core/src/index.ts"))); + expect(result.symbols[0]?.file).toBe("packages/core/src/index.ts"); + }); +}); + +describe("normalize — privacy (defensive filter)", () => { + it("skips member with isPrivate flag", () => { + const result = normalize(project( + cls("Foo", "src/foo.ts", privateFlag(prop("secret", "src/foo.ts"))) + )); + expect(result.symbols.map(s => s.name)).not.toContain("Foo.secret"); + }); + + it("skips member with isProtected flag", () => { + const result = normalize(project( + cls("Foo", "src/foo.ts", protectedFlag(prop("internal", "src/foo.ts"))) + )); + expect(result.symbols.map(s => s.name)).not.toContain("Foo.internal"); + }); +}); + +// ── Fixture tests ──────────────────────────────────────────────────────────── +// These run against real TypeDoc 0.27 output generated from the minimal +// TS project in test/fixtures/typedoc-sample.json. + +describe("normalize (fixture — real TypeDoc 0.27 output)", () => { + const fixture = JSON.parse( + readFileSync(join(__dirname, "fixtures/typedoc-sample.json"), "utf8") + ); + + it("finds AuthClient class", () => { + const names = normalize(fixture).symbols.map(s => s.name); + expect(names).toContain("AuthClient"); + }); + + it("finds AuthClient.signUp method", () => { + const names = normalize(fixture).symbols.map(s => s.name); + expect(names).toContain("AuthClient.signUp"); + }); + + it("finds Session interface as class kind", () => { + const sym = normalize(fixture).symbols.find(s => s.name === "Session"); + expect(sym?.kind).toBe("class"); + }); + + it("finds UserRole enum as class kind", () => { + const sym = normalize(fixture).symbols.find(s => s.name === "UserRole"); + expect(sym?.kind).toBe("class"); + }); + + it("finds AuthResponse type alias as variable kind", () => { + const sym = normalize(fixture).symbols.find(s => s.name === "AuthResponse"); + expect(sym?.kind).toBe("variable"); + }); + + it("finds createClient function", () => { + const sym = normalize(fixture).symbols.find(s => s.name === "createClient"); + expect(sym?.kind).toBe("function"); + }); + + it("finds VERSION variable", () => { + const sym = normalize(fixture).symbols.find(s => s.name === "VERSION"); + expect(sym?.kind).toBe("variable"); + }); + + it("does not emit Client (re-export Reference)", () => { + const names = normalize(fixture).symbols.map(s => s.name); + expect(names).not.toContain("Client"); + }); + + it("does not emit constructor", () => { + const names = normalize(fixture).symbols.map(s => s.name); + expect(names).not.toContain("AuthClient.constructor"); + }); + + it("does not emit private _cache field", () => { + const names = normalize(fixture).symbols.map(s => s.name); + expect(names).not.toContain("AuthClient._cache"); + }); +}); +``` + +- [ ] **Step 2: Run the tests — expect failure** + +```bash +cd scripts/capability-matrix +npx vitest run test/normalize-typedoc.test.ts 2>&1 | head -20 +``` + +Expected: `Cannot find module '../src/normalize-typedoc.js'` + +- [ ] **Step 3: Implement normalize-typedoc.ts** + +Create `scripts/capability-matrix/src/normalize-typedoc.ts`: + +```typescript +export interface ParsedSymbol { + name: string; + kind: "class" | "method" | "property" | "function" | "variable"; + file: string; +} + +export interface ParseResult { + symbols: ParsedSymbol[]; +} + +// TypeDoc ReflectionKind numeric constants used by the normalizer. +const Kind = { + Module: 2, + Namespace: 4, + Enum: 8, + EnumMember: 16, + Variable: 32, + Function: 64, + Class: 128, + Interface: 256, + Constructor: 512, + Property: 1024, + Method: 2048, + Accessor: 262144, + TypeAlias: 2097152, + Reference: 4194304, +} as const; + +interface TdReflection { + name: string; + kind: number; + flags?: { isPrivate?: boolean; isProtected?: boolean }; + sources?: Array<{ fileName: string }>; + children?: TdReflection[]; +} + +function fileOf(r: TdReflection): string { + return r.sources?.[0]?.fileName ?? ""; +} + +function isExcluded(r: TdReflection): boolean { + return !!(r.flags?.isPrivate || r.flags?.isProtected); +} + +function extractMembers( + parent: string, + children: TdReflection[], + out: ParsedSymbol[], +): void { + for (const child of children) { + if (isExcluded(child)) continue; + if (child.kind === Kind.Constructor) continue; + const qualName = `${parent}.${child.name}`; + const file = fileOf(child); + if (child.kind === Kind.Method) { + out.push({ name: qualName, kind: "method", file }); + } else if (child.kind === Kind.Property) { + out.push({ name: qualName, kind: "property", file }); + } else if (child.kind === Kind.Accessor) { + out.push({ name: qualName, kind: "method", file }); + } else if (child.kind === Kind.EnumMember) { + out.push({ name: qualName, kind: "property", file }); + } + } +} + +function extractDeclarations( + children: TdReflection[], + out: ParsedSymbol[], +): void { + for (const child of children) { + if (isExcluded(child)) continue; + const file = fileOf(child); + if (child.kind === Kind.Module || child.kind === Kind.Namespace) { + if (child.children) extractDeclarations(child.children, out); + } else if (child.kind === Kind.Reference) { + continue; + } else if ( + child.kind === Kind.Class || + child.kind === Kind.Interface || + child.kind === Kind.Enum + ) { + out.push({ name: child.name, kind: "class", file }); + if (child.children) extractMembers(child.name, child.children, out); + } else if (child.kind === Kind.Function) { + out.push({ name: child.name, kind: "function", file }); + } else if (child.kind === Kind.Variable || child.kind === Kind.TypeAlias) { + out.push({ name: child.name, kind: "variable", file }); + } + } +} + +export function normalize(json: unknown): ParseResult { + const project = json as TdReflection; + const symbols: ParsedSymbol[] = []; + if (project.children) extractDeclarations(project.children, symbols); + return { symbols }; +} +``` + +- [ ] **Step 4: Update swift-parser.ts to import from normalize-typedoc.ts** + +In `scripts/capability-matrix/src/swift-parser.ts`, find these two lines at the top: + +```typescript +import type { ParsedSymbol, ParseResult } from "./ts-parser.js"; +export type { ParsedSymbol, ParseResult }; +``` + +Change to: + +```typescript +import type { ParsedSymbol, ParseResult } from "./normalize-typedoc.js"; +export type { ParsedSymbol, ParseResult }; +``` + +- [ ] **Step 5: Run the tests — expect all pass** + +```bash +cd scripts/capability-matrix +npx vitest run test/normalize-typedoc.test.ts +``` + +Expected: All tests PASS. If fixture tests fail with "AuthClient not found", re-run Task 1 Step 4 to confirm the fixture shape. + +- [ ] **Step 6: Run the full suite to confirm no regressions** + +```bash +cd scripts/capability-matrix +npm test +``` + +Expected: All tests pass. `ts-parser.test.ts` may fail (it's deleted in Task 5 — that's expected). + +- [ ] **Step 7: Commit** + +```bash +git add scripts/capability-matrix/src/normalize-typedoc.ts \ + scripts/capability-matrix/src/swift-parser.ts \ + scripts/capability-matrix/test/normalize-typedoc.test.ts +git commit -m "feat: add normalize-typedoc with TypeDoc JSON → ParseResult mapping" +``` + +--- + +### Task 3: CLI wrapper and package.json + +**Files:** +- Create: `scripts/capability-matrix/src/normalize-typedoc-cli.ts` +- Modify: `scripts/capability-matrix/package.json` + +**Interfaces:** +- Consumes: `normalize(json: unknown): ParseResult` from `./normalize-typedoc.js` +- Produces: `normalize-typedoc` script; invoked as `npm run normalize-typedoc -- ` + +- [ ] **Step 1: Write the CLI** + +Create `scripts/capability-matrix/src/normalize-typedoc-cli.ts`: + +```typescript +import { readFileSync, writeFileSync } from "node:fs"; +import { normalize } from "./normalize-typedoc.js"; + +const [, , inputPath, outputPath] = process.argv; + +if (!inputPath || !outputPath) { + console.error("Usage: normalize-typedoc "); + process.exit(1); +} + +const json = JSON.parse(readFileSync(inputPath, "utf8")); +const result = normalize(json); +writeFileSync(outputPath, JSON.stringify(result, null, 2)); +``` + +- [ ] **Step 2: Update package.json** + +In `scripts/capability-matrix/package.json`, inside `"scripts"`: + +Replace: +```json +"parse-ts": "tsx src/parse-ts.ts", +``` + +With: +```json +"normalize-typedoc": "tsx src/normalize-typedoc-cli.ts", +``` + +- [ ] **Step 3: Smoke-test the CLI** + +```bash +cd scripts/capability-matrix +npm run normalize-typedoc -- test/fixtures/typedoc-sample.json /tmp/ts-symbols.json +node -e "const r = JSON.parse(require('fs').readFileSync('/tmp/ts-symbols.json','utf8')); console.log(r.symbols.slice(0,5))" +``` + +Expected: Array of 5 `ParsedSymbol` objects with names like `AuthClient`, `AuthClient.signUp`, etc. + +- [ ] **Step 4: Commit** + +```bash +git add scripts/capability-matrix/src/normalize-typedoc-cli.ts \ + scripts/capability-matrix/package.json +git commit -m "feat: add normalize-typedoc-cli and update package.json" +``` + +--- + +### Task 4: Update validate-sdk-compliance.yml + +**Files:** +- Modify: `.github/workflows/validate-sdk-compliance.yml` + +**Interfaces:** +- Produces: TypeScript-conditional steps that replace the `parse-ts` invocation; base/PR symbol files produced at `$GITHUB_WORKSPACE/pr-symbols.json` and `$GITHUB_WORKSPACE/base-symbols.json` (same paths the existing `check-api-symbols` step reads) + +- [ ] **Step 1: Add the entrypoint input** + +In the `workflow_call.inputs` block, add after the existing `sdk-ref` input: + +```yaml + entrypoint: + description: TypeScript entrypoint file relative to SDK root (TypeScript SDKs only) + type: string + default: src/index.ts +``` + +- [ ] **Step 2: Add TypeScript-conditional steps before the existing "Resolve parse command" step** + +In the `check` job, insert the following steps between `Install dependencies` and `Resolve parse command`: + +```yaml + - name: Install PR SDK dependencies (TypeScript) + if: inputs.language == 'javascript' + run: npm ci + working-directory: _sdk-pr + + - name: Generate TypeDoc JSON — PR branch + if: inputs.language == 'javascript' + run: | + npx --yes typedoc@0.27 \ + --json "$GITHUB_WORKSPACE/pr-raw.json" \ + --excludePrivate --excludeProtected \ + ${{ inputs.entrypoint }} + working-directory: _sdk-pr + + - name: Install base SDK dependencies (TypeScript) + if: inputs.language == 'javascript' + run: npm ci + working-directory: _sdk-base + + - name: Generate TypeDoc JSON — base branch + if: inputs.language == 'javascript' + run: | + npx --yes typedoc@0.27 \ + --json "$GITHUB_WORKSPACE/base-raw.json" \ + --excludePrivate --excludeProtected \ + ${{ inputs.entrypoint }} + working-directory: _sdk-base + + - name: Normalize TypeDoc output (TypeScript) + if: inputs.language == 'javascript' + run: | + npm run --silent normalize-typedoc -- \ + "$GITHUB_WORKSPACE/pr-raw.json" "$GITHUB_WORKSPACE/pr-symbols.json" + npm run --silent normalize-typedoc -- \ + "$GITHUB_WORKSPACE/base-raw.json" "$GITHUB_WORKSPACE/base-symbols.json" + working-directory: _sdk-spec/scripts/capability-matrix +``` + +- [ ] **Step 3: Gate the existing parse steps on non-javascript** + +The existing `Resolve parse command`, `Parse PR branch`, and `Parse base branch` steps currently run for all languages. Add `if: inputs.language != 'javascript'` to each: + +```yaml + - name: Resolve parse command + if: inputs.language != 'javascript' + id: resolve + run: | + case "${{ inputs.language }}" in + swift) echo "cmd=parse-swift" >> "$GITHUB_OUTPUT" ;; + *) echo "::error::Unsupported language '${{ inputs.language }}'. Supported values: swift, javascript (javascript uses typedoc path)"; exit 1 ;; + esac + + - name: Parse PR branch + if: inputs.language != 'javascript' + run: | + npm run --silent ${{ steps.resolve.outputs.cmd }} -- "$GITHUB_WORKSPACE/_sdk-pr" \ + > "$GITHUB_WORKSPACE/pr-symbols.json" + working-directory: _sdk-spec/scripts/capability-matrix + + - name: Parse base branch + if: inputs.language != 'javascript' + run: | + npm run --silent ${{ steps.resolve.outputs.cmd }} -- "$GITHUB_WORKSPACE/_sdk-base" \ + > "$GITHUB_WORKSPACE/base-symbols.json" + working-directory: _sdk-spec/scripts/capability-matrix +``` + +The `Check new symbols against capability matrix` step has no `if` condition — it runs for both languages and reads `pr-symbols.json` / `base-symbols.json` regardless of which path produced them. + +- [ ] **Step 4: Validate the YAML** + +```bash +npx js-yaml .github/workflows/validate-sdk-compliance.yml > /dev/null && echo "YAML valid" +``` + +Expected: `YAML valid` + +- [ ] **Step 5: Commit** + +```bash +git add .github/workflows/validate-sdk-compliance.yml +git commit -m "ci: replace parse-ts with typedoc + normalize-typedoc in validate-sdk-compliance" +``` + +--- + +### Task 5: Delete old files and final verification + +**Files:** +- Delete: `scripts/capability-matrix/src/ts-parser.ts` +- Delete: `scripts/capability-matrix/src/parse-ts.ts` +- Delete: `scripts/capability-matrix/test/ts-parser.test.ts` +- Delete: `scripts/capability-matrix/test/fixtures/ts-sample/` + +- [ ] **Step 1: Confirm nothing still imports ts-parser** + +```bash +cd scripts/capability-matrix +grep -r "ts-parser" src/ test/ --include="*.ts" -l +``` + +Expected: No output. If any file appears, update its import to reference `normalize-typedoc.js` instead. + +- [ ] **Step 2: Delete the old files** + +```bash +rm scripts/capability-matrix/src/ts-parser.ts +rm scripts/capability-matrix/src/parse-ts.ts +rm scripts/capability-matrix/test/ts-parser.test.ts +rm -rf scripts/capability-matrix/test/fixtures/ts-sample/ +``` + +- [ ] **Step 3: Run the full test suite** + +```bash +cd scripts/capability-matrix +npm test +``` + +Expected: All tests pass with no failures or type errors. + +- [ ] **Step 4: Run typecheck** + +```bash +cd scripts/capability-matrix +npm run typecheck +``` + +Expected: No errors. If `ts-parser` is still referenced in a type import somewhere, fix the import and re-run. + +- [ ] **Step 5: Commit** + +```bash +git add -u +git commit -m "refactor: remove ts-parser and parse-ts in favor of normalize-typedoc" +``` From f4b531bb1a0abce5a74bdd9550cb5e91288768de Mon Sep 17 00:00:00 2001 From: Guilherme Souza Date: Fri, 19 Jun 2026 11:06:53 -0300 Subject: [PATCH 3/9] test: add real typedoc-sample.json fixture for normalizer tests --- .../test/fixtures/typedoc-sample.json | 675 ++++++++++++++++++ 1 file changed, 675 insertions(+) create mode 100644 scripts/capability-matrix/test/fixtures/typedoc-sample.json diff --git a/scripts/capability-matrix/test/fixtures/typedoc-sample.json b/scripts/capability-matrix/test/fixtures/typedoc-sample.json new file mode 100644 index 0000000..5ad5e8e --- /dev/null +++ b/scripts/capability-matrix/test/fixtures/typedoc-sample.json @@ -0,0 +1,675 @@ +{ + "id": 0, + "name": "fixture", + "variant": "project", + "kind": 1, + "flags": {}, + "children": [ + { + "id": 20, + "name": "UserRole", + "variant": "declaration", + "kind": 8, + "flags": {}, + "children": [ + { + "id": 21, + "name": "Admin", + "variant": "declaration", + "kind": 16, + "flags": {}, + "sources": [ + { + "fileName": "index.ts", + "line": 15, + "character": 2 + } + ], + "type": { + "type": "literal", + "value": "admin" + } + }, + { + "id": 22, + "name": "User", + "variant": "declaration", + "kind": 16, + "flags": {}, + "sources": [ + { + "fileName": "index.ts", + "line": 16, + "character": 2 + } + ], + "type": { + "type": "literal", + "value": "user" + } + } + ], + "groups": [ + { + "title": "Enumeration Members", + "children": [ + 21, + 22 + ] + } + ], + "sources": [ + { + "fileName": "index.ts", + "line": 14, + "character": 12 + } + ] + }, + { + "id": 4, + "name": "AuthClient", + "variant": "declaration", + "kind": 128, + "flags": {}, + "children": [ + { + "id": 5, + "name": "constructor", + "variant": "declaration", + "kind": 512, + "flags": {}, + "sources": [ + { + "fileName": "index.ts", + "line": 2, + "character": 2 + } + ], + "signatures": [ + { + "id": 6, + "name": "AuthClient", + "variant": "signature", + "kind": 16384, + "flags": {}, + "sources": [ + { + "fileName": "index.ts", + "line": 2, + "character": 2 + } + ], + "parameters": [ + { + "id": 7, + "name": "url", + "variant": "param", + "kind": 32768, + "flags": {}, + "type": { + "type": "intrinsic", + "name": "string" + } + } + ], + "type": { + "type": "reference", + "target": 4, + "name": "AuthClient", + "package": "fixture" + } + } + ] + }, + { + "id": 14, + "name": "session", + "variant": "declaration", + "kind": 262144, + "flags": {}, + "sources": [ + { + "fileName": "index.ts", + "line": 5, + "character": 6 + } + ], + "getSignature": { + "id": 15, + "name": "session", + "variant": "signature", + "kind": 524288, + "flags": {}, + "sources": [ + { + "fileName": "index.ts", + "line": 5, + "character": 6 + } + ], + "type": { + "type": "intrinsic", + "name": "string" + } + } + }, + { + "id": 12, + "name": "signIn", + "variant": "declaration", + "kind": 2048, + "flags": {}, + "sources": [ + { + "fileName": "index.ts", + "line": 4, + "character": 2 + } + ], + "signatures": [ + { + "id": 13, + "name": "signIn", + "variant": "signature", + "kind": 4096, + "flags": {}, + "sources": [ + { + "fileName": "index.ts", + "line": 4, + "character": 2 + } + ], + "type": { + "type": "intrinsic", + "name": "void" + } + } + ] + }, + { + "id": 9, + "name": "signUp", + "variant": "declaration", + "kind": 2048, + "flags": {}, + "sources": [ + { + "fileName": "index.ts", + "line": 3, + "character": 2 + } + ], + "signatures": [ + { + "id": 10, + "name": "signUp", + "variant": "signature", + "kind": 4096, + "flags": {}, + "sources": [ + { + "fileName": "index.ts", + "line": 3, + "character": 2 + } + ], + "parameters": [ + { + "id": 11, + "name": "email", + "variant": "param", + "kind": 32768, + "flags": {}, + "type": { + "type": "intrinsic", + "name": "string" + } + } + ], + "type": { + "type": "intrinsic", + "name": "void" + } + } + ] + } + ], + "groups": [ + { + "title": "Constructors", + "children": [ + 5 + ] + }, + { + "title": "Accessors", + "children": [ + 14 + ] + }, + { + "title": "Methods", + "children": [ + 12, + 9 + ] + } + ], + "sources": [ + { + "fileName": "index.ts", + "line": 1, + "character": 13 + } + ] + }, + { + "id": 17, + "name": "Session", + "variant": "declaration", + "kind": 256, + "flags": {}, + "children": [ + { + "id": 19, + "name": "expires", + "variant": "declaration", + "kind": 1024, + "flags": {}, + "sources": [ + { + "fileName": "index.ts", + "line": 11, + "character": 2 + } + ], + "type": { + "type": "reference", + "target": { + "sourceFileName": "node_modules/typescript/lib/lib.es5.d.ts", + "qualifiedName": "Date" + }, + "name": "Date", + "package": "typescript" + } + }, + { + "id": 18, + "name": "user", + "variant": "declaration", + "kind": 1024, + "flags": {}, + "sources": [ + { + "fileName": "index.ts", + "line": 10, + "character": 2 + } + ], + "type": { + "type": "intrinsic", + "name": "string" + } + } + ], + "groups": [ + { + "title": "Properties", + "children": [ + 19, + 18 + ] + } + ], + "sources": [ + { + "fileName": "index.ts", + "line": 9, + "character": 17 + } + ] + }, + { + "id": 23, + "name": "AuthResponse", + "variant": "declaration", + "kind": 2097152, + "flags": {}, + "sources": [ + { + "fileName": "index.ts", + "line": 19, + "character": 12 + } + ], + "type": { + "type": "reflection", + "declaration": { + "id": 24, + "name": "__type", + "variant": "declaration", + "kind": 65536, + "flags": {}, + "children": [ + { + "id": 25, + "name": "data", + "variant": "declaration", + "kind": 1024, + "flags": {}, + "sources": [ + { + "fileName": "index.ts", + "line": 19, + "character": 29 + } + ], + "type": { + "type": "intrinsic", + "name": "unknown" + } + }, + { + "id": 26, + "name": "error", + "variant": "declaration", + "kind": 1024, + "flags": {}, + "sources": [ + { + "fileName": "index.ts", + "line": 19, + "character": 44 + } + ], + "type": { + "type": "union", + "types": [ + { + "type": "reference", + "target": { + "sourceFileName": "node_modules/typescript/lib/lib.es5.d.ts", + "qualifiedName": "Error" + }, + "name": "Error", + "package": "typescript" + }, + { + "type": "literal", + "value": null + } + ] + } + } + ], + "groups": [ + { + "title": "Properties", + "children": [ + 25, + 26 + ] + } + ], + "sources": [ + { + "fileName": "index.ts", + "line": 19, + "character": 27 + } + ] + } + } + }, + { + "id": 27, + "name": "VERSION", + "variant": "declaration", + "kind": 32, + "flags": { + "isConst": true + }, + "sources": [ + { + "fileName": "index.ts", + "line": 25, + "character": 13 + } + ], + "type": { + "type": "literal", + "value": "1.0.0" + }, + "defaultValue": "\"1.0.0\"" + }, + { + "id": 1, + "name": "createClient", + "variant": "declaration", + "kind": 64, + "flags": {}, + "sources": [ + { + "fileName": "index.ts", + "line": 21, + "character": 16 + } + ], + "signatures": [ + { + "id": 2, + "name": "createClient", + "variant": "signature", + "kind": 4096, + "flags": {}, + "sources": [ + { + "fileName": "index.ts", + "line": 21, + "character": 16 + } + ], + "parameters": [ + { + "id": 3, + "name": "url", + "variant": "param", + "kind": 32768, + "flags": {}, + "type": { + "type": "intrinsic", + "name": "string" + } + } + ], + "type": { + "type": "reference", + "target": 4, + "name": "AuthClient", + "package": "fixture" + } + } + ] + }, + { + "id": 28, + "name": "Client", + "variant": "reference", + "kind": 4194304, + "flags": {}, + "sources": [ + { + "fileName": "index.ts", + "line": 27, + "character": 23 + } + ], + "target": 4 + } + ], + "groups": [ + { + "title": "Enumerations", + "children": [ + 20 + ] + }, + { + "title": "Classes", + "children": [ + 4 + ] + }, + { + "title": "Interfaces", + "children": [ + 17 + ] + }, + { + "title": "Type Aliases", + "children": [ + 23 + ] + }, + { + "title": "Variables", + "children": [ + 27 + ] + }, + { + "title": "Functions", + "children": [ + 1 + ] + }, + { + "title": "References", + "children": [ + 28 + ] + } + ], + "packageName": "fixture", + "symbolIdMap": { + "0": { + "sourceFileName": "src/index.ts", + "qualifiedName": "" + }, + "1": { + "sourceFileName": "src/index.ts", + "qualifiedName": "createClient" + }, + "2": { + "sourceFileName": "src/index.ts", + "qualifiedName": "createClient" + }, + "3": { + "sourceFileName": "src/index.ts", + "qualifiedName": "url" + }, + "4": { + "sourceFileName": "src/index.ts", + "qualifiedName": "AuthClient" + }, + "5": { + "sourceFileName": "src/index.ts", + "qualifiedName": "AuthClient.__constructor" + }, + "6": { + "sourceFileName": "src/index.ts", + "qualifiedName": "AuthClient" + }, + "7": { + "sourceFileName": "src/index.ts", + "qualifiedName": "url" + }, + "9": { + "sourceFileName": "src/index.ts", + "qualifiedName": "AuthClient.signUp" + }, + "10": { + "sourceFileName": "src/index.ts", + "qualifiedName": "AuthClient.signUp" + }, + "11": { + "sourceFileName": "src/index.ts", + "qualifiedName": "email" + }, + "12": { + "sourceFileName": "src/index.ts", + "qualifiedName": "AuthClient.signIn" + }, + "13": { + "sourceFileName": "src/index.ts", + "qualifiedName": "AuthClient.signIn" + }, + "14": { + "sourceFileName": "src/index.ts", + "qualifiedName": "AuthClient.session" + }, + "15": { + "sourceFileName": "src/index.ts", + "qualifiedName": "AuthClient.session" + }, + "17": { + "sourceFileName": "src/index.ts", + "qualifiedName": "Session" + }, + "18": { + "sourceFileName": "src/index.ts", + "qualifiedName": "Session.user" + }, + "19": { + "sourceFileName": "src/index.ts", + "qualifiedName": "Session.expires" + }, + "20": { + "sourceFileName": "src/index.ts", + "qualifiedName": "UserRole" + }, + "21": { + "sourceFileName": "src/index.ts", + "qualifiedName": "UserRole.Admin" + }, + "22": { + "sourceFileName": "src/index.ts", + "qualifiedName": "UserRole.User" + }, + "23": { + "sourceFileName": "src/index.ts", + "qualifiedName": "AuthResponse" + }, + "24": { + "sourceFileName": "src/index.ts", + "qualifiedName": "__type" + }, + "25": { + "sourceFileName": "src/index.ts", + "qualifiedName": "__type.data" + }, + "26": { + "sourceFileName": "src/index.ts", + "qualifiedName": "__type.error" + }, + "27": { + "sourceFileName": "src/index.ts", + "qualifiedName": "VERSION" + }, + "28": { + "sourceFileName": "src/index.ts", + "qualifiedName": "Client" + } + }, + "files": { + "entries": { + "1": "src/index.ts" + }, + "reflections": { + "1": 0 + } + } +} From 59b8c8394dcb863f67caf5192755addbd30c8a01 Mon Sep 17 00:00:00 2001 From: Guilherme Souza Date: Fri, 19 Jun 2026 11:12:08 -0300 Subject: [PATCH 4/9] =?UTF-8?q?feat:=20add=20normalize-typedoc=20with=20Ty?= =?UTF-8?q?peDoc=20JSON=20=E2=86=92=20ParseResult=20mapping?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .../src/normalize-typedoc.ts | 97 ++++++++ .../test/normalize-typedoc.test.ts | 217 ++++++++++++++++++ 2 files changed, 314 insertions(+) create mode 100644 scripts/capability-matrix/src/normalize-typedoc.ts create mode 100644 scripts/capability-matrix/test/normalize-typedoc.test.ts diff --git a/scripts/capability-matrix/src/normalize-typedoc.ts b/scripts/capability-matrix/src/normalize-typedoc.ts new file mode 100644 index 0000000..00770ef --- /dev/null +++ b/scripts/capability-matrix/src/normalize-typedoc.ts @@ -0,0 +1,97 @@ +export interface ParsedSymbol { + name: string; + kind: "class" | "method" | "property" | "function" | "variable"; + file: string; +} + +export interface ParseResult { + symbols: ParsedSymbol[]; +} + +const Kind = { + Module: 2, + Namespace: 4, + Enum: 8, + EnumMember: 16, + Variable: 32, + Function: 64, + Class: 128, + Interface: 256, + Constructor: 512, + Property: 1024, + Method: 2048, + Accessor: 262144, + TypeAlias: 2097152, + Reference: 4194304, +} as const; + +interface TdReflection { + name: string; + kind: number; + flags?: { isPrivate?: boolean; isProtected?: boolean }; + sources?: Array<{ fileName: string }>; + children?: TdReflection[]; +} + +function fileOf(r: TdReflection): string { + return r.sources?.[0]?.fileName ?? ""; +} + +function isExcluded(r: TdReflection): boolean { + return !!(r.flags?.isPrivate || r.flags?.isProtected); +} + +function extractMembers( + parent: string, + children: TdReflection[], + out: ParsedSymbol[], +): void { + for (const child of children) { + if (isExcluded(child)) continue; + if (child.kind === Kind.Constructor) continue; + const qualName = `${parent}.${child.name}`; + const file = fileOf(child); + if (child.kind === Kind.Method) { + out.push({ name: qualName, kind: "method", file }); + } else if (child.kind === Kind.Property) { + out.push({ name: qualName, kind: "property", file }); + } else if (child.kind === Kind.Accessor) { + out.push({ name: qualName, kind: "method", file }); + } else if (child.kind === Kind.EnumMember) { + out.push({ name: qualName, kind: "property", file }); + } + } +} + +function extractDeclarations( + children: TdReflection[], + out: ParsedSymbol[], +): void { + for (const child of children) { + if (isExcluded(child)) continue; + const file = fileOf(child); + if (child.kind === Kind.Module || child.kind === Kind.Namespace) { + if (child.children) extractDeclarations(child.children, out); + } else if (child.kind === Kind.Reference) { + continue; + } else if ( + child.kind === Kind.Class || + child.kind === Kind.Interface || + child.kind === Kind.Enum + ) { + out.push({ name: child.name, kind: "class", file }); + if (child.children) extractMembers(child.name, child.children, out); + } else if (child.kind === Kind.Function) { + out.push({ name: child.name, kind: "function", file }); + } else if (child.kind === Kind.Variable || child.kind === Kind.TypeAlias) { + out.push({ name: child.name, kind: "variable", file }); + } + } +} + +export function normalize(json: unknown): ParseResult { + const project = json as TdReflection; + const symbols: ParsedSymbol[] = []; + if (project.children) extractDeclarations(project.children, symbols); + return { symbols }; +} diff --git a/scripts/capability-matrix/test/normalize-typedoc.test.ts b/scripts/capability-matrix/test/normalize-typedoc.test.ts new file mode 100644 index 0000000..8fdfa50 --- /dev/null +++ b/scripts/capability-matrix/test/normalize-typedoc.test.ts @@ -0,0 +1,217 @@ +import { describe, it, expect } from "vitest"; +import { join, dirname } from "node:path"; +import { fileURLToPath } from "node:url"; +import { readFileSync } from "node:fs"; +import { normalize } from "../src/normalize-typedoc.js"; + +const __dirname = dirname(fileURLToPath(import.meta.url)); + +function project(...children: object[]) { + return { kind: 1, name: "test", children }; +} +function mod(name: string, ...children: object[]) { + return { kind: 2, name, flags: {}, children }; +} +function cls(name: string, file: string, ...members: object[]) { + return { kind: 128, name, flags: {}, sources: [{ fileName: file }], children: members }; +} +function iface(name: string, file: string, ...members: object[]) { + return { kind: 256, name, flags: {}, sources: [{ fileName: file }], children: members }; +} +function enumDecl(name: string, file: string, ...members: object[]) { + return { kind: 8, name, flags: {}, sources: [{ fileName: file }], children: members }; +} +function method(name: string, file: string) { + return { kind: 2048, name, flags: {}, sources: [{ fileName: file }] }; +} +function prop(name: string, file: string) { + return { kind: 1024, name, flags: {}, sources: [{ fileName: file }] }; +} +function accessor(name: string, file: string) { + return { kind: 262144, name, flags: {}, sources: [{ fileName: file }] }; +} +function ctor(file: string) { + return { kind: 512, name: "constructor", flags: {}, sources: [{ fileName: file }] }; +} +function enumMember(name: string, file: string) { + return { kind: 16, name, flags: {}, sources: [{ fileName: file }] }; +} +function fn(name: string, file: string) { + return { kind: 64, name, flags: {}, sources: [{ fileName: file }] }; +} +function variable(name: string, file: string) { + return { kind: 32, name, flags: {}, sources: [{ fileName: file }] }; +} +function typeAlias(name: string, file: string) { + return { kind: 2097152, name, flags: {}, sources: [{ fileName: file }] }; +} +function ref(name: string) { + return { kind: 4194304, name, flags: {} }; +} +function privateFlag(base: object): object { + return { ...base, flags: { isPrivate: true } }; +} +function protectedFlag(base: object): object { + return { ...base, flags: { isProtected: true } }; +} + +describe("normalize — class", () => { + it("emits class symbol", () => { + const result = normalize(project(cls("AuthClient", "src/auth.ts"))); + expect(result.symbols).toContainEqual({ name: "AuthClient", kind: "class", file: "src/auth.ts" }); + }); + + it("emits class method", () => { + const result = normalize(project(cls("AuthClient", "src/auth.ts", method("signUp", "src/auth.ts")))); + expect(result.symbols).toContainEqual({ name: "AuthClient.signUp", kind: "method", file: "src/auth.ts" }); + }); + + it("emits class property", () => { + const result = normalize(project(cls("AuthClient", "src/auth.ts", prop("session", "src/auth.ts")))); + expect(result.symbols).toContainEqual({ name: "AuthClient.session", kind: "property", file: "src/auth.ts" }); + }); + + it("emits accessor as method kind", () => { + const result = normalize(project(cls("AuthClient", "src/auth.ts", accessor("token", "src/auth.ts")))); + expect(result.symbols).toContainEqual({ name: "AuthClient.token", kind: "method", file: "src/auth.ts" }); + }); + + it("skips constructor", () => { + const result = normalize(project(cls("Foo", "src/foo.ts", ctor("src/foo.ts"), method("bar", "src/foo.ts")))); + const names = result.symbols.map(s => s.name); + expect(names).not.toContain("Foo.constructor"); + expect(names).toContain("Foo.bar"); + }); +}); + +describe("normalize — interface", () => { + it("emits interface as class kind", () => { + const result = normalize(project(iface("Session", "src/session.ts"))); + expect(result.symbols).toContainEqual({ name: "Session", kind: "class", file: "src/session.ts" }); + }); + + it("emits interface members as property", () => { + const result = normalize(project(iface("Session", "src/session.ts", prop("user", "src/session.ts")))); + expect(result.symbols).toContainEqual({ name: "Session.user", kind: "property", file: "src/session.ts" }); + }); +}); + +describe("normalize — enum", () => { + it("emits enum as class kind", () => { + const result = normalize(project(enumDecl("UserRole", "src/role.ts"))); + expect(result.symbols).toContainEqual({ name: "UserRole", kind: "class", file: "src/role.ts" }); + }); + + it("emits enum member as property kind", () => { + const result = normalize(project(enumDecl("UserRole", "src/role.ts", enumMember("Admin", "src/role.ts")))); + expect(result.symbols).toContainEqual({ name: "UserRole.Admin", kind: "property", file: "src/role.ts" }); + }); +}); + +describe("normalize — top-level declarations", () => { + it("emits exported function", () => { + const result = normalize(project(fn("createClient", "src/index.ts"))); + expect(result.symbols).toContainEqual({ name: "createClient", kind: "function", file: "src/index.ts" }); + }); + + it("emits exported variable", () => { + const result = normalize(project(variable("VERSION", "src/index.ts"))); + expect(result.symbols).toContainEqual({ name: "VERSION", kind: "variable", file: "src/index.ts" }); + }); + + it("emits type alias as variable kind", () => { + const result = normalize(project(typeAlias("AuthResponse", "src/types.ts"))); + expect(result.symbols).toContainEqual({ name: "AuthResponse", kind: "variable", file: "src/types.ts" }); + }); + + it("skips Reference kind", () => { + const result = normalize(project(cls("AuthClient", "src/auth.ts"), ref("Client"))); + const names = result.symbols.map(s => s.name); + expect(names).not.toContain("Client"); + expect(names).toContain("AuthClient"); + }); +}); + +describe("normalize — traversal", () => { + it("walks into Module wrapper (kind 2)", () => { + const result = normalize(project(mod("src/auth", cls("AuthClient", "src/auth.ts")))); + expect(result.symbols).toContainEqual({ name: "AuthClient", kind: "class", file: "src/auth.ts" }); + }); + + it("captures file path from sources[0].fileName", () => { + const result = normalize(project(fn("foo", "packages/core/src/index.ts"))); + expect(result.symbols[0]?.file).toBe("packages/core/src/index.ts"); + }); +}); + +describe("normalize — privacy (defensive filter)", () => { + it("skips member with isPrivate flag", () => { + const result = normalize(project( + cls("Foo", "src/foo.ts", privateFlag(prop("secret", "src/foo.ts"))) + )); + expect(result.symbols.map(s => s.name)).not.toContain("Foo.secret"); + }); + + it("skips member with isProtected flag", () => { + const result = normalize(project( + cls("Foo", "src/foo.ts", protectedFlag(prop("internal", "src/foo.ts"))) + )); + expect(result.symbols.map(s => s.name)).not.toContain("Foo.internal"); + }); +}); + +describe("normalize (fixture — real TypeDoc 0.27 output)", () => { + const fixture = JSON.parse( + readFileSync(join(__dirname, "fixtures/typedoc-sample.json"), "utf8") + ); + + it("finds AuthClient class", () => { + const names = normalize(fixture).symbols.map(s => s.name); + expect(names).toContain("AuthClient"); + }); + + it("finds AuthClient.signUp method", () => { + const names = normalize(fixture).symbols.map(s => s.name); + expect(names).toContain("AuthClient.signUp"); + }); + + it("finds Session interface as class kind", () => { + const sym = normalize(fixture).symbols.find(s => s.name === "Session"); + expect(sym?.kind).toBe("class"); + }); + + it("finds UserRole enum as class kind", () => { + const sym = normalize(fixture).symbols.find(s => s.name === "UserRole"); + expect(sym?.kind).toBe("class"); + }); + + it("finds AuthResponse type alias as variable kind", () => { + const sym = normalize(fixture).symbols.find(s => s.name === "AuthResponse"); + expect(sym?.kind).toBe("variable"); + }); + + it("finds createClient function", () => { + const sym = normalize(fixture).symbols.find(s => s.name === "createClient"); + expect(sym?.kind).toBe("function"); + }); + + it("finds VERSION variable", () => { + const sym = normalize(fixture).symbols.find(s => s.name === "VERSION"); + expect(sym?.kind).toBe("variable"); + }); + + it("does not emit Client (re-export Reference)", () => { + const names = normalize(fixture).symbols.map(s => s.name); + expect(names).not.toContain("Client"); + }); + + it("does not emit constructor", () => { + const names = normalize(fixture).symbols.map(s => s.name); + expect(names).not.toContain("AuthClient.constructor"); + }); + + it("does not emit private _cache field", () => { + const names = normalize(fixture).symbols.map(s => s.name); + expect(names).not.toContain("AuthClient._cache"); + }); +}); From 7985169ff237973f7e021559b7c4b76b9a2abda7 Mon Sep 17 00:00:00 2001 From: Guilherme Souza Date: Fri, 19 Jun 2026 11:16:20 -0300 Subject: [PATCH 5/9] feat: add normalize-typedoc-cli and update package.json --- scripts/capability-matrix/package.json | 2 +- .../capability-matrix/src/normalize-typedoc-cli.ts | 13 +++++++++++++ 2 files changed, 14 insertions(+), 1 deletion(-) create mode 100644 scripts/capability-matrix/src/normalize-typedoc-cli.ts diff --git a/scripts/capability-matrix/package.json b/scripts/capability-matrix/package.json index 8eb0dbb..2f75e49 100644 --- a/scripts/capability-matrix/package.json +++ b/scripts/capability-matrix/package.json @@ -8,7 +8,7 @@ "validate": "tsx src/cli.ts validate", "validate:online": "tsx src/cli.ts validate --online", "validate-compliance": "tsx src/compliance-cli.ts", - "parse-ts": "tsx src/parse-ts.ts", + "normalize-typedoc": "tsx src/normalize-typedoc-cli.ts", "normalize-symbolgraph": "tsx src/normalize-symbolgraph-cli.ts", "normalize-griffe": "tsx src/normalize-griffe-cli.ts", "check-api-symbols": "tsx src/check-api-symbols.ts", diff --git a/scripts/capability-matrix/src/normalize-typedoc-cli.ts b/scripts/capability-matrix/src/normalize-typedoc-cli.ts new file mode 100644 index 0000000..ff6e054 --- /dev/null +++ b/scripts/capability-matrix/src/normalize-typedoc-cli.ts @@ -0,0 +1,13 @@ +import { readFileSync, writeFileSync } from "node:fs"; +import { normalize } from "./normalize-typedoc.js"; + +const [, , inputPath, outputPath] = process.argv; + +if (!inputPath || !outputPath) { + console.error("Usage: normalize-typedoc "); + process.exit(1); +} + +const json = JSON.parse(readFileSync(inputPath, "utf8")); +const result = normalize(json); +writeFileSync(outputPath, JSON.stringify(result, null, 2)); From b42a2388d0a9b657e85e92d2896a597301e868eb Mon Sep 17 00:00:00 2001 From: Guilherme Souza Date: Fri, 19 Jun 2026 11:18:52 -0300 Subject: [PATCH 6/9] ci: replace parse-ts with typedoc + normalize-typedoc in validate-sdk-compliance --- .github/workflows/validate-sdk-compliance.yml | 50 +++++++++++++++---- 1 file changed, 41 insertions(+), 9 deletions(-) diff --git a/.github/workflows/validate-sdk-compliance.yml b/.github/workflows/validate-sdk-compliance.yml index aae8d2d..dd0ddfe 100644 --- a/.github/workflows/validate-sdk-compliance.yml +++ b/.github/workflows/validate-sdk-compliance.yml @@ -23,6 +23,10 @@ on: description: Comma-separated search paths for griffe relative to repo root (python only) type: string default: "" + entrypoint: + description: TypeScript entrypoint file relative to SDK root (TypeScript SDKs only) + type: string + default: src/index.ts jobs: validate: @@ -99,6 +103,43 @@ jobs: run: npm ci working-directory: _sdk-spec/scripts/capability-matrix + - name: Install PR SDK dependencies (TypeScript) + if: inputs.language == 'javascript' + run: npm ci + working-directory: _sdk-pr + + - name: Generate TypeDoc JSON — PR branch + if: inputs.language == 'javascript' + run: | + npx --yes typedoc@0.27 \ + --json "$GITHUB_WORKSPACE/pr-raw.json" \ + --excludePrivate --excludeProtected \ + "${{ inputs.entrypoint }}" + working-directory: _sdk-pr + + - name: Install base SDK dependencies (TypeScript) + if: inputs.language == 'javascript' + run: npm ci + working-directory: _sdk-base + + - name: Generate TypeDoc JSON — base branch + if: inputs.language == 'javascript' + run: | + npx --yes typedoc@0.27 \ + --json "$GITHUB_WORKSPACE/base-raw.json" \ + --excludePrivate --excludeProtected \ + "${{ inputs.entrypoint }}" + working-directory: _sdk-base + + - name: Normalize TypeDoc output (TypeScript) + if: inputs.language == 'javascript' + run: | + npm run --silent normalize-typedoc -- \ + "$GITHUB_WORKSPACE/pr-raw.json" "$GITHUB_WORKSPACE/pr-symbols.json" + npm run --silent normalize-typedoc -- \ + "$GITHUB_WORKSPACE/base-raw.json" "$GITHUB_WORKSPACE/base-symbols.json" + working-directory: _sdk-spec/scripts/capability-matrix + - name: Set up Python if: inputs.language == 'python' uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0 @@ -184,15 +225,6 @@ jobs: done working-directory: _sdk-spec/scripts/capability-matrix - - name: Parse JavaScript symbols - if: inputs.language == 'javascript' - run: | - for b in pr base; do - npm run --silent parse-ts -- "$GITHUB_WORKSPACE/_sdk-$b" \ - > "$GITHUB_WORKSPACE/$b-symbols.json" - done - working-directory: _sdk-spec/scripts/capability-matrix - - name: Check new symbols against capability matrix run: | npm run check-api-symbols -- \ From f92df8578feba327faa6900e78b019bcb70007d6 Mon Sep 17 00:00:00 2001 From: Guilherme Souza Date: Fri, 19 Jun 2026 11:22:49 -0300 Subject: [PATCH 7/9] refactor: remove ts-parser and parse-ts in favor of normalize-typedoc --- scripts/capability-matrix/src/api-check.ts | 2 +- .../src/check-api-symbols.ts | 2 +- scripts/capability-matrix/src/parse-ts.ts | 19 --- scripts/capability-matrix/src/ts-parser.ts | 137 ------------------ .../capability-matrix/test/api-check.test.ts | 2 +- .../test/fixtures/ts-sample/src/index.ts | 38 ----- .../capability-matrix/test/ts-parser.test.ts | 135 ----------------- 7 files changed, 3 insertions(+), 332 deletions(-) delete mode 100644 scripts/capability-matrix/src/parse-ts.ts delete mode 100644 scripts/capability-matrix/src/ts-parser.ts delete mode 100644 scripts/capability-matrix/test/fixtures/ts-sample/src/index.ts delete mode 100644 scripts/capability-matrix/test/ts-parser.test.ts diff --git a/scripts/capability-matrix/src/api-check.ts b/scripts/capability-matrix/src/api-check.ts index 7952508..316a9db 100644 --- a/scripts/capability-matrix/src/api-check.ts +++ b/scripts/capability-matrix/src/api-check.ts @@ -1,6 +1,6 @@ import { buildSymbolIndex } from "./compliance.js"; import type { RawCompliance } from "./compliance.js"; -import type { ParsedSymbol } from "./ts-parser.js"; +import type { ParsedSymbol } from "./normalize-typedoc.js"; export interface CheckResult { newSymbols: string[]; diff --git a/scripts/capability-matrix/src/check-api-symbols.ts b/scripts/capability-matrix/src/check-api-symbols.ts index 62ebe1f..11fe4b9 100644 --- a/scripts/capability-matrix/src/check-api-symbols.ts +++ b/scripts/capability-matrix/src/check-api-symbols.ts @@ -3,7 +3,7 @@ import { resolve } from "node:path"; import { parse } from "yaml"; import { checkNewSymbols, formatErrorMessage, formatRemovedMessage } from "./api-check.js"; import type { RawCompliance } from "./compliance.js"; -import type { ParseResult } from "./ts-parser.js"; +import type { ParseResult } from "./normalize-typedoc.js"; async function main(): Promise { const [prFile, baseFile, compliancePath] = process.argv.slice(2); diff --git a/scripts/capability-matrix/src/parse-ts.ts b/scripts/capability-matrix/src/parse-ts.ts deleted file mode 100644 index ce92c70..0000000 --- a/scripts/capability-matrix/src/parse-ts.ts +++ /dev/null @@ -1,19 +0,0 @@ -import { parseTypeScriptProject } from "./ts-parser.js"; - -async function main(): Promise { - const projectPath = process.argv[2]; - if (!projectPath) { - console.error("Usage: parse-ts "); - process.exit(1); - } - - try { - const result = parseTypeScriptProject(projectPath); - console.log(JSON.stringify(result, null, 2)); - } catch (e) { - console.error(`Error: ${(e as Error).message}`); - process.exit(1); - } -} - -main().catch((e) => { console.error(e); process.exit(1); }); diff --git a/scripts/capability-matrix/src/ts-parser.ts b/scripts/capability-matrix/src/ts-parser.ts deleted file mode 100644 index cbe6d31..0000000 --- a/scripts/capability-matrix/src/ts-parser.ts +++ /dev/null @@ -1,137 +0,0 @@ -import ts from "typescript"; -import { readFileSync, readdirSync, existsSync } from "node:fs"; -import { join, resolve, relative } from "node:path"; -import { loadIgnore, type Ignore } from "./parse-ignore.js"; - -export interface ParsedSymbol { - name: string; - kind: "class" | "method" | "property" | "function" | "variable"; - file: string; -} - -export interface ParseResult { - symbols: ParsedSymbol[]; -} - -// --------------------------------------------------------------------------- -// Internal helpers -// --------------------------------------------------------------------------- - -function findSourceFiles(dir: string, root: string, ig: Ignore): string[] { - const results: string[] = []; - try { - for (const entry of readdirSync(dir, { withFileTypes: true })) { - if (entry.name.startsWith(".")) continue; - const full = join(dir, entry.name); - const rel = relative(root, full); - if (entry.isDirectory()) { - if (ig.ignores(rel + "/")) continue; - results.push(...findSourceFiles(full, root, ig)); - } else if (entry.isFile() && entry.name.endsWith(".ts")) { - if (ig.ignores(rel)) continue; - results.push(full); - } - } - } catch { /* ignore unreadable dirs */ } - return results; -} - -function hasModifier(node: ts.Node, kind: ts.SyntaxKind): boolean { - if (!ts.canHaveModifiers(node)) return false; - const mods = ts.getModifiers(node); - return mods?.some((m) => m.kind === kind) ?? false; -} - -function isExported(node: ts.Node): boolean { - return hasModifier(node, ts.SyntaxKind.ExportKeyword); -} - -function isPublicMember(member: ts.ClassElement): boolean { - if (member.name?.kind === ts.SyntaxKind.PrivateIdentifier) return false; - if (hasModifier(member, ts.SyntaxKind.PrivateKeyword)) return false; - if (hasModifier(member, ts.SyntaxKind.ProtectedKeyword)) return false; - return true; -} - -function memberIdentifierName(member: ts.ClassElement): string | undefined { - const n = member.name; - if (!n) return undefined; - if (ts.isIdentifier(n)) return n.text; - if (ts.isStringLiteral(n)) return n.text; - return undefined; -} - -function extractClassMembers( - className: string, - node: ts.ClassDeclaration, - relPath: string, - out: ParsedSymbol[], -): void { - for (const member of node.members) { - if (ts.isConstructorDeclaration(member)) continue; - if (!isPublicMember(member)) continue; - - const name = memberIdentifierName(member); - if (!name) continue; - - const kind = - ts.isMethodDeclaration(member) || - ts.isGetAccessorDeclaration(member) || - ts.isSetAccessorDeclaration(member) - ? "method" - : "property"; - - out.push({ name: `${className}.${name}`, kind, file: relPath }); - } -} - -// --------------------------------------------------------------------------- -// Public API -// --------------------------------------------------------------------------- - -export function extractFromSource( - source: string, - relPath: string, -): ParsedSymbol[] { - const sf = ts.createSourceFile(relPath, source, ts.ScriptTarget.Latest, true); - const symbols: ParsedSymbol[] = []; - - for (const stmt of sf.statements) { - if (ts.isClassDeclaration(stmt) && isExported(stmt)) { - const className = stmt.name?.text; - if (className) { - symbols.push({ name: className, kind: "class", file: relPath }); - extractClassMembers(className, stmt, relPath, symbols); - } - } else if (ts.isFunctionDeclaration(stmt) && isExported(stmt)) { - const name = stmt.name?.text; - if (name) symbols.push({ name, kind: "function", file: relPath }); - } else if (ts.isVariableStatement(stmt) && isExported(stmt)) { - for (const decl of stmt.declarationList.declarations) { - if (ts.isIdentifier(decl.name)) { - symbols.push({ name: decl.name.text, kind: "variable", file: relPath }); - } - } - } - } - - return symbols; -} - -export function parseTypeScriptProject(projectRoot: string): ParseResult { - const root = resolve(projectRoot); - const ig = loadIgnore(root); - const srcDir = join(root, "src"); - const scanRoot = existsSync(srcDir) ? srcDir : root; - - const files = findSourceFiles(scanRoot, root, ig); - const symbols: ParsedSymbol[] = []; - - for (const file of files) { - const source = readFileSync(file, "utf8"); - const relPath = relative(root, file); - symbols.push(...extractFromSource(source, relPath)); - } - - return { symbols }; -} diff --git a/scripts/capability-matrix/test/api-check.test.ts b/scripts/capability-matrix/test/api-check.test.ts index 4fc60f1..bdb5e40 100644 --- a/scripts/capability-matrix/test/api-check.test.ts +++ b/scripts/capability-matrix/test/api-check.test.ts @@ -1,6 +1,6 @@ import { describe, it, expect } from "vitest"; import { checkNewSymbols, formatErrorMessage, formatRemovedMessage } from "../src/api-check"; -import type { ParsedSymbol } from "../src/ts-parser"; +import type { ParsedSymbol } from "../src/normalize-typedoc"; function sym(name: string): ParsedSymbol { return { name, kind: "method", file: "src/index.ts" }; diff --git a/scripts/capability-matrix/test/fixtures/ts-sample/src/index.ts b/scripts/capability-matrix/test/fixtures/ts-sample/src/index.ts deleted file mode 100644 index c49da69..0000000 --- a/scripts/capability-matrix/test/fixtures/ts-sample/src/index.ts +++ /dev/null @@ -1,38 +0,0 @@ -export class AuthClient { - public signUp(email: string, password: string): Promise { - return Promise.resolve(); - } - - public signIn(email: string): Promise { - return Promise.resolve(); - } - - get session(): string | null { - return null; - } - - private _token: string | null = null; - - protected _refresh(): void {} - - #privateField = "hidden"; -} - -export class StorageClient { - public upload(path: string): Promise { - return Promise.resolve(); - } -} - -export function createClient(url: string, key: string): AuthClient { - return new AuthClient(); -} - -export const version = "1.0.0"; - -// Not exported — must not appear in output -class InternalHelper { - public doSomething(): void {} -} - -function internalUtil(): void {} diff --git a/scripts/capability-matrix/test/ts-parser.test.ts b/scripts/capability-matrix/test/ts-parser.test.ts deleted file mode 100644 index 648cbea..0000000 --- a/scripts/capability-matrix/test/ts-parser.test.ts +++ /dev/null @@ -1,135 +0,0 @@ -import { describe, it, expect } from "vitest"; -import { join, dirname } from "node:path"; -import { fileURLToPath } from "node:url"; -import { tmpdir } from "node:os"; -import { writeFileSync, cpSync } from "node:fs"; -import { extractFromSource, parseTypeScriptProject } from "../src/ts-parser"; - -const __dirname = dirname(fileURLToPath(import.meta.url)); -const FIXTURE = join(__dirname, "fixtures", "ts-sample"); - -describe("extractFromSource", () => { - it("extracts exported class and its public methods", () => { - const source = ` - export class AuthClient { - public signUp(email: string): void {} - public signIn(): void {} - } - `; - const symbols = extractFromSource(source, "src/index.ts"); - const names = symbols.map((s) => s.name); - expect(names).toContain("AuthClient"); - expect(names).toContain("AuthClient.signUp"); - expect(names).toContain("AuthClient.signIn"); - }); - - it("excludes private and protected members", () => { - const source = ` - export class Foo { - public pub(): void {} - private priv(): void {} - protected prot(): void {} - #hard = 1; - } - `; - const symbols = extractFromSource(source, "src/index.ts"); - const names = symbols.map((s) => s.name); - expect(names).toContain("Foo.pub"); - expect(names).not.toContain("Foo.priv"); - expect(names).not.toContain("Foo.prot"); - expect(names).not.toContain("Foo.#hard"); - }); - - it("excludes non-exported classes", () => { - const source = ` - class Internal { - public method(): void {} - } - `; - const symbols = extractFromSource(source, "src/index.ts"); - expect(symbols).toHaveLength(0); - }); - - it("extracts exported functions", () => { - const source = `export function createClient(url: string): void {}`; - const symbols = extractFromSource(source, "src/index.ts"); - expect(symbols).toEqual([{ name: "createClient", kind: "function", file: "src/index.ts" }]); - }); - - it("extracts exported variables", () => { - const source = `export const version = "1.0.0";`; - const symbols = extractFromSource(source, "src/index.ts"); - expect(symbols).toEqual([{ name: "version", kind: "variable", file: "src/index.ts" }]); - }); - - it("skips constructor", () => { - const source = ` - export class Foo { - constructor(private x: number) {} - public bar(): void {} - } - `; - const symbols = extractFromSource(source, "src/index.ts"); - const names = symbols.map((s) => s.name); - expect(names).not.toContain("Foo.constructor"); - expect(names).toContain("Foo.bar"); - }); - - it("includes getter as property kind", () => { - const source = ` - export class Foo { - get session(): string { return ""; } - } - `; - const symbols = extractFromSource(source, "src/index.ts"); - const s = symbols.find((x) => x.name === "Foo.session"); - expect(s).toBeDefined(); - expect(s?.kind).toBe("method"); - }); -}); - -describe("parseTypeScriptProject (fixture)", () => { - it("parses the fixture project and finds expected symbols", () => { - const result = parseTypeScriptProject(FIXTURE); - const names = result.symbols.map((s) => s.name); - - expect(names).toContain("AuthClient"); - expect(names).toContain("AuthClient.signUp"); - expect(names).toContain("AuthClient.signIn"); - expect(names).toContain("AuthClient.session"); - expect(names).toContain("StorageClient"); - expect(names).toContain("StorageClient.upload"); - expect(names).toContain("createClient"); - expect(names).toContain("version"); - }); - - it("excludes private and internal symbols from fixture", () => { - const result = parseTypeScriptProject(FIXTURE); - const names = result.symbols.map((s) => s.name); - - expect(names).not.toContain("AuthClient._token"); - expect(names).not.toContain("AuthClient._refresh"); - expect(names).not.toContain("InternalHelper"); - expect(names).not.toContain("internalUtil"); - }); -}); - -describe("parseTypeScriptProject — .sdk-parse-ignore", () => { - it("excludes files matched by .sdk-parse-ignore", () => { - // Copy fixture to a temp dir so we can add an ignore file without - // polluting the committed fixture. - const dir = join(tmpdir(), `ts-parser-ignore-test-${process.pid}`); - cpSync(FIXTURE, dir, { recursive: true }); - // The fixture has src/index.ts which exports AuthClient. - // Ignore the entire src/ directory. - writeFileSync(join(dir, ".sdk-parse-ignore"), "src/\n"); - const result = parseTypeScriptProject(dir); - expect(result.symbols).toHaveLength(0); - }); - - it("does not exclude files when .sdk-parse-ignore is absent", () => { - // FIXTURE has no .sdk-parse-ignore — should parse normally. - const result = parseTypeScriptProject(FIXTURE); - expect(result.symbols.map((s) => s.name)).toContain("AuthClient"); - }); -}); From 71048e8e820cd788ba7b2b23f7ba947c2bd3f4cc Mon Sep 17 00:00:00 2001 From: Guilherme Souza Date: Fri, 19 Jun 2026 11:32:27 -0300 Subject: [PATCH 8/9] =?UTF-8?q?fix:=20address=20final=20review=20findings?= =?UTF-8?q?=20=E2=80=94=20docs,=20entrypoint=20quoting,=20tests?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - CLAUDE.md: remove stale ts-parser/parse-ts rows; add normalize-typedoc.ts and normalize-typedoc-cli.ts; clarify parse-ignore.ts is Swift-only - docs/specs: add Behavioral Notes section explaining .sdk-parse-ignore no longer applies to TypeScript (TypeDoc uses entrypoint resolution instead) - validate-sdk-compliance.yml: quote \${{ inputs.entrypoint }} in both TypeDoc run steps to handle paths with spaces - normalize-typedoc.test.ts: replace vacuous _cache test with meaningful accessor fixture test; add Namespace (kind 4) traversal test --- CLAUDE.md | 7 +++---- docs/specs/2026-06-19-ts-parser-typedoc-design.md | 6 ++++++ .../capability-matrix/test/normalize-typedoc.test.ts | 12 +++++++++--- 3 files changed, 18 insertions(+), 7 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 50455f9..11781d1 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -59,12 +59,11 @@ capabilities/*.yaml → validate (AJV schema) → aggregate (GitHub API fetc - `aggregate.ts` — Fetches compliance files from all SDK repos via Octokit - `generate-site.ts` — Builds the static HTML matrix site - `report.ts` — Calculates parity percentages per feature/area/language -- `ts-parser.ts` — Syntactic TypeScript AST walker; extracts public symbols without requiring `node_modules` - `swift-parser.ts` — Line-by-line Swift scanner; extracts public/open symbols from classes, structs, actors, enums, extensions -- `parse-ts.ts` — CLI wrapper for `ts-parser.ts`; takes an SDK root path and emits `ParseResult` JSON -- `parse-swift.ts` — CLI wrapper for `swift-parser.ts`; same contract as `parse-ts.ts` +- `normalize-typedoc.ts` — TypeDoc JSON normalizer; maps TypeDoc reflection kinds to `ParseResult`; defines `ParsedSymbol` and `ParseResult` types +- `normalize-typedoc-cli.ts` — CLI wrapper; reads TypeDoc JSON, calls normalizer, writes `ParseResult` JSON - `scripts/dart_symbol_extractor/` (sibling Dart package) — Small `package:analyzer` tool that walks `lib/**.dart` syntactically and emits the same `ParseResult` JSON; run directly with `dart run bin/extract.dart `. Parses without `pub get`; supports extension types and enhanced enums -- `parse-ignore.ts` — Loads `.sdk-parse-ignore` (gitignore syntax) to exclude paths from symbol parsing +- `parse-ignore.ts` — Loads `.sdk-parse-ignore` (gitignore syntax) to exclude paths from Swift symbol parsing (TypeScript uses TypeDoc entrypoint resolution instead) - `api-check.ts` — Diff logic: `checkNewSymbols(base, pr, compliance)` returns symbols added in PR not in the compliance file - `check-api-symbols.ts` — CLI; compares two `ParseResult` files against `sdk-compliance.yaml`, exits 1 with a clear error on uncovered symbols diff --git a/docs/specs/2026-06-19-ts-parser-typedoc-design.md b/docs/specs/2026-06-19-ts-parser-typedoc-design.md index 497fea7..707d5bf 100644 --- a/docs/specs/2026-06-19-ts-parser-typedoc-design.md +++ b/docs/specs/2026-06-19-ts-parser-typedoc-design.md @@ -106,6 +106,12 @@ TypeDoc is pinned at `0.27` and invoked via `npx --yes typedoc@0.27` — no chan The `language: javascript` input value is unchanged; SDK repos that already call this workflow need no modifications. +## Behavioral Notes + +### `.sdk-parse-ignore` no longer applies to TypeScript + +The old `ts-parser.ts` honored `.sdk-parse-ignore` (gitignore-syntax file) to suppress specific file paths from its output. The TypeDoc path does not use `.sdk-parse-ignore`. TypeDoc naturally limits its output to symbols reachable from the configured entrypoint — internal files and test paths that are not re-exported through the package's `exports` field are not included in the TypeDoc JSON and therefore are not emitted by the normalizer. `.sdk-parse-ignore` continues to apply to Swift parsing via `swift-parser.ts` and `parse-ignore.ts`. + ## Testing The fixture (`typedoc-sample.json`) is generated from a minimal TypeScript project that exercises all mapped cases. It is committed to the repo so tests are hermetic and do not require a TypeScript build at test time. diff --git a/scripts/capability-matrix/test/normalize-typedoc.test.ts b/scripts/capability-matrix/test/normalize-typedoc.test.ts index 8fdfa50..5e17b83 100644 --- a/scripts/capability-matrix/test/normalize-typedoc.test.ts +++ b/scripts/capability-matrix/test/normalize-typedoc.test.ts @@ -138,6 +138,12 @@ describe("normalize — traversal", () => { expect(result.symbols).toContainEqual({ name: "AuthClient", kind: "class", file: "src/auth.ts" }); }); + it("walks into Namespace wrapper (kind 4)", () => { + const ns = { kind: 4, name: "Utils", flags: {}, children: [fn("helper", "src/utils.ts")] }; + const result = normalize(project(ns)); + expect(result.symbols).toContainEqual({ name: "helper", kind: "function", file: "src/utils.ts" }); + }); + it("captures file path from sources[0].fileName", () => { const result = normalize(project(fn("foo", "packages/core/src/index.ts"))); expect(result.symbols[0]?.file).toBe("packages/core/src/index.ts"); @@ -210,8 +216,8 @@ describe("normalize (fixture — real TypeDoc 0.27 output)", () => { expect(names).not.toContain("AuthClient.constructor"); }); - it("does not emit private _cache field", () => { - const names = normalize(fixture).symbols.map(s => s.name); - expect(names).not.toContain("AuthClient._cache"); + it("emits AuthClient.session accessor as method kind from fixture", () => { + const sym = normalize(fixture).symbols.find(s => s.name === "AuthClient.session"); + expect(sym?.kind).toBe("method"); }); }); From de55fe816d93299ea0803d9abb5adbc64c51eda8 Mon Sep 17 00:00:00 2001 From: Guilherme Souza Date: Fri, 19 Jun 2026 11:41:30 -0300 Subject: [PATCH 9/9] chore: move design docs to docs/superpowers (gitignored) --- docs/plans/2026-06-19-ts-parser-typedoc.md | 754 ------------------ .../2026-06-19-ts-parser-typedoc-design.md | 137 ---- 2 files changed, 891 deletions(-) delete mode 100644 docs/plans/2026-06-19-ts-parser-typedoc.md delete mode 100644 docs/specs/2026-06-19-ts-parser-typedoc-design.md diff --git a/docs/plans/2026-06-19-ts-parser-typedoc.md b/docs/plans/2026-06-19-ts-parser-typedoc.md deleted file mode 100644 index d5382ab..0000000 --- a/docs/plans/2026-06-19-ts-parser-typedoc.md +++ /dev/null @@ -1,754 +0,0 @@ -# TypeScript Parser TypeDoc Refactor Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** Replace `ts-parser.ts` (hand-rolled TypeScript AST walker) with a thin normalizer over TypeDoc 0.27 JSON output, following the same "external tool → normalizer → ParseResult" architecture as the Dart parser added in PR #35. - -**Architecture:** TypeDoc 0.27 runs in CI inside the SDK repo (after `npm ci`), producing a JSON file of all public declarations. `normalize-typedoc.ts` maps TypeDoc's reflection kinds to the existing `ParseResult` shape. `check-api-symbols` and everything downstream are untouched. `ParsedSymbol`/`ParseResult` type definitions move from `ts-parser.ts` to `normalize-typedoc.ts`; `swift-parser.ts` re-imports from there. - -**Tech Stack:** TypeScript 5, TypeDoc 0.27, Vitest 4, Node 22, `tsx` for dev scripts - -## Global Constraints - -- TypeDoc pinned at `0.27` — invoked as `npx --yes typedoc@0.27` -- `ParsedSymbol.kind` stays `"class" | "method" | "property" | "function" | "variable"` — no new values -- `check-api-symbols.ts` and `api-check.ts` are not modified -- All existing tests must pass after cleanup -- Conventional commits: `feat:`, `refactor:`, `test:`, `ci:` -- All scripts in `package.json` use `tsx` prefix (matches existing pattern) -- Working directory for capability-matrix commands: `scripts/capability-matrix/` - ---- - -### Task 1: Generate typedoc-sample.json fixture - -**Files:** -- Create: `scripts/capability-matrix/test/fixtures/typedoc-sample.json` - -**Interfaces:** -- Produces: real TypeDoc 0.27 JSON consumed by fixture-based tests in Task 2 - -- [ ] **Step 1: Create a minimal TypeScript project in a temp directory** - -```bash -mkdir -p /tmp/typedoc-fixture/src -``` - -Write `/tmp/typedoc-fixture/src/index.ts`: -```typescript -export class AuthClient { - constructor(private url: string) {} - signUp(email: string): void {} - signIn(): void {} - get session(): string { return ""; } - private _cache = new Map(); -} - -export interface Session { - user: string; - expires: Date; -} - -export enum UserRole { - Admin = "admin", - User = "user", -} - -export type AuthResponse = { data: unknown; error: Error | null }; - -export function createClient(url: string): AuthClient { - return new AuthClient(url); -} - -export const VERSION = "1.0.0"; - -export { AuthClient as Client }; -``` - -Write `/tmp/typedoc-fixture/tsconfig.json`: -```json -{ - "compilerOptions": { - "target": "ES2020", - "module": "NodeNext", - "moduleResolution": "NodeNext", - "strict": true - }, - "include": ["src"] -} -``` - -Write `/tmp/typedoc-fixture/package.json`: -```json -{ "name": "fixture", "version": "1.0.0", "type": "module" } -``` - -- [ ] **Step 2: Install TypeDoc and generate JSON** - -```bash -cd /tmp/typedoc-fixture -npm install --save-dev typedoc@0.27 -npx typedoc --json api.json --excludePrivate --excludeProtected src/index.ts -``` - -Expected: `api.json` is created. TypeDoc may emit warnings — these are fine as long as the file is produced. - -- [ ] **Step 3: Copy the fixture into the repo** - -```bash -cp /tmp/typedoc-fixture/api.json \ - /scripts/capability-matrix/test/fixtures/typedoc-sample.json -``` - -Replace `` with the actual absolute path to the repo. - -- [ ] **Step 4: Sanity-check the fixture** - -```bash -cd scripts/capability-matrix -node --input-type=module <<'EOF' -import { readFileSync } from "fs"; -const f = JSON.parse(readFileSync("test/fixtures/typedoc-sample.json", "utf8")); -const walk = (children) => children.flatMap(c => c.kind === 2 ? walk(c.children ?? []) : [c]); -console.log(walk(f.children ?? []).map(c => `${c.kind} ${c.name}`)); -EOF -``` - -Expected output includes lines like `128 AuthClient`, `256 Session`, `8 UserRole`, `64 createClient`, `32 VERSION`, `2097152 AuthResponse` and a Reference line for `Client`. - -- [ ] **Step 5: Commit** - -```bash -git add scripts/capability-matrix/test/fixtures/typedoc-sample.json -git commit -m "test: add real typedoc-sample.json fixture for normalizer tests" -``` - ---- - -### Task 2: TDD — write tests, implement normalize-typedoc.ts - -**Files:** -- Create: `scripts/capability-matrix/src/normalize-typedoc.ts` -- Create: `scripts/capability-matrix/test/normalize-typedoc.test.ts` -- Modify: `scripts/capability-matrix/src/swift-parser.ts` (update import path only) - -**Interfaces:** -- Produces: `normalize(json: unknown): ParseResult` exported from `normalize-typedoc.ts` -- Produces: `ParsedSymbol`, `ParseResult` types exported from `normalize-typedoc.ts` (replacing `ts-parser.ts` as the canonical source) - -- [ ] **Step 1: Write the failing tests** - -Create `scripts/capability-matrix/test/normalize-typedoc.test.ts`: - -```typescript -import { describe, it, expect } from "vitest"; -import { join, dirname } from "node:path"; -import { fileURLToPath } from "node:url"; -import { readFileSync } from "node:fs"; -import { normalize } from "../src/normalize-typedoc.js"; - -const __dirname = dirname(fileURLToPath(import.meta.url)); - -// ── Inline JSON helpers ────────────────────────────────────────────────────── -// Each builds the minimal TypeDoc shape needed for a specific case. - -function project(...children: object[]) { - return { kind: 1, name: "test", children }; -} -function mod(name: string, ...children: object[]) { - return { kind: 2, name, flags: {}, children }; -} -function cls(name: string, file: string, ...members: object[]) { - return { kind: 128, name, flags: {}, sources: [{ fileName: file }], children: members }; -} -function iface(name: string, file: string, ...members: object[]) { - return { kind: 256, name, flags: {}, sources: [{ fileName: file }], children: members }; -} -function enumDecl(name: string, file: string, ...members: object[]) { - return { kind: 8, name, flags: {}, sources: [{ fileName: file }], children: members }; -} -function method(name: string, file: string) { - return { kind: 2048, name, flags: {}, sources: [{ fileName: file }] }; -} -function prop(name: string, file: string) { - return { kind: 1024, name, flags: {}, sources: [{ fileName: file }] }; -} -function accessor(name: string, file: string) { - return { kind: 262144, name, flags: {}, sources: [{ fileName: file }] }; -} -function ctor(file: string) { - return { kind: 512, name: "constructor", flags: {}, sources: [{ fileName: file }] }; -} -function enumMember(name: string, file: string) { - return { kind: 16, name, flags: {}, sources: [{ fileName: file }] }; -} -function fn(name: string, file: string) { - return { kind: 64, name, flags: {}, sources: [{ fileName: file }] }; -} -function variable(name: string, file: string) { - return { kind: 32, name, flags: {}, sources: [{ fileName: file }] }; -} -function typeAlias(name: string, file: string) { - return { kind: 2097152, name, flags: {}, sources: [{ fileName: file }] }; -} -function ref(name: string) { - return { kind: 4194304, name, flags: {} }; -} -function privateFlag(base: object): object { - return { ...base, flags: { isPrivate: true } }; -} -function protectedFlag(base: object): object { - return { ...base, flags: { isProtected: true } }; -} - -// ── Unit tests (inline JSON) ───────────────────────────────────────────────── - -describe("normalize — class", () => { - it("emits class symbol", () => { - const result = normalize(project(cls("AuthClient", "src/auth.ts"))); - expect(result.symbols).toContainEqual({ name: "AuthClient", kind: "class", file: "src/auth.ts" }); - }); - - it("emits class method", () => { - const result = normalize(project(cls("AuthClient", "src/auth.ts", method("signUp", "src/auth.ts")))); - expect(result.symbols).toContainEqual({ name: "AuthClient.signUp", kind: "method", file: "src/auth.ts" }); - }); - - it("emits class property", () => { - const result = normalize(project(cls("AuthClient", "src/auth.ts", prop("session", "src/auth.ts")))); - expect(result.symbols).toContainEqual({ name: "AuthClient.session", kind: "property", file: "src/auth.ts" }); - }); - - it("emits accessor as method kind", () => { - const result = normalize(project(cls("AuthClient", "src/auth.ts", accessor("token", "src/auth.ts")))); - expect(result.symbols).toContainEqual({ name: "AuthClient.token", kind: "method", file: "src/auth.ts" }); - }); - - it("skips constructor", () => { - const result = normalize(project(cls("Foo", "src/foo.ts", ctor("src/foo.ts"), method("bar", "src/foo.ts")))); - const names = result.symbols.map(s => s.name); - expect(names).not.toContain("Foo.constructor"); - expect(names).toContain("Foo.bar"); - }); -}); - -describe("normalize — interface", () => { - it("emits interface as class kind", () => { - const result = normalize(project(iface("Session", "src/session.ts"))); - expect(result.symbols).toContainEqual({ name: "Session", kind: "class", file: "src/session.ts" }); - }); - - it("emits interface members as property", () => { - const result = normalize(project(iface("Session", "src/session.ts", prop("user", "src/session.ts")))); - expect(result.symbols).toContainEqual({ name: "Session.user", kind: "property", file: "src/session.ts" }); - }); -}); - -describe("normalize — enum", () => { - it("emits enum as class kind", () => { - const result = normalize(project(enumDecl("UserRole", "src/role.ts"))); - expect(result.symbols).toContainEqual({ name: "UserRole", kind: "class", file: "src/role.ts" }); - }); - - it("emits enum member as property kind", () => { - const result = normalize(project(enumDecl("UserRole", "src/role.ts", enumMember("Admin", "src/role.ts")))); - expect(result.symbols).toContainEqual({ name: "UserRole.Admin", kind: "property", file: "src/role.ts" }); - }); -}); - -describe("normalize — top-level declarations", () => { - it("emits exported function", () => { - const result = normalize(project(fn("createClient", "src/index.ts"))); - expect(result.symbols).toContainEqual({ name: "createClient", kind: "function", file: "src/index.ts" }); - }); - - it("emits exported variable", () => { - const result = normalize(project(variable("VERSION", "src/index.ts"))); - expect(result.symbols).toContainEqual({ name: "VERSION", kind: "variable", file: "src/index.ts" }); - }); - - it("emits type alias as variable kind", () => { - const result = normalize(project(typeAlias("AuthResponse", "src/types.ts"))); - expect(result.symbols).toContainEqual({ name: "AuthResponse", kind: "variable", file: "src/types.ts" }); - }); - - it("skips Reference kind", () => { - const result = normalize(project(cls("AuthClient", "src/auth.ts"), ref("Client"))); - const names = result.symbols.map(s => s.name); - expect(names).not.toContain("Client"); - expect(names).toContain("AuthClient"); - }); -}); - -describe("normalize — traversal", () => { - it("walks into Module wrapper (kind 2)", () => { - const result = normalize(project(mod("src/auth", cls("AuthClient", "src/auth.ts")))); - expect(result.symbols).toContainEqual({ name: "AuthClient", kind: "class", file: "src/auth.ts" }); - }); - - it("captures file path from sources[0].fileName", () => { - const result = normalize(project(fn("foo", "packages/core/src/index.ts"))); - expect(result.symbols[0]?.file).toBe("packages/core/src/index.ts"); - }); -}); - -describe("normalize — privacy (defensive filter)", () => { - it("skips member with isPrivate flag", () => { - const result = normalize(project( - cls("Foo", "src/foo.ts", privateFlag(prop("secret", "src/foo.ts"))) - )); - expect(result.symbols.map(s => s.name)).not.toContain("Foo.secret"); - }); - - it("skips member with isProtected flag", () => { - const result = normalize(project( - cls("Foo", "src/foo.ts", protectedFlag(prop("internal", "src/foo.ts"))) - )); - expect(result.symbols.map(s => s.name)).not.toContain("Foo.internal"); - }); -}); - -// ── Fixture tests ──────────────────────────────────────────────────────────── -// These run against real TypeDoc 0.27 output generated from the minimal -// TS project in test/fixtures/typedoc-sample.json. - -describe("normalize (fixture — real TypeDoc 0.27 output)", () => { - const fixture = JSON.parse( - readFileSync(join(__dirname, "fixtures/typedoc-sample.json"), "utf8") - ); - - it("finds AuthClient class", () => { - const names = normalize(fixture).symbols.map(s => s.name); - expect(names).toContain("AuthClient"); - }); - - it("finds AuthClient.signUp method", () => { - const names = normalize(fixture).symbols.map(s => s.name); - expect(names).toContain("AuthClient.signUp"); - }); - - it("finds Session interface as class kind", () => { - const sym = normalize(fixture).symbols.find(s => s.name === "Session"); - expect(sym?.kind).toBe("class"); - }); - - it("finds UserRole enum as class kind", () => { - const sym = normalize(fixture).symbols.find(s => s.name === "UserRole"); - expect(sym?.kind).toBe("class"); - }); - - it("finds AuthResponse type alias as variable kind", () => { - const sym = normalize(fixture).symbols.find(s => s.name === "AuthResponse"); - expect(sym?.kind).toBe("variable"); - }); - - it("finds createClient function", () => { - const sym = normalize(fixture).symbols.find(s => s.name === "createClient"); - expect(sym?.kind).toBe("function"); - }); - - it("finds VERSION variable", () => { - const sym = normalize(fixture).symbols.find(s => s.name === "VERSION"); - expect(sym?.kind).toBe("variable"); - }); - - it("does not emit Client (re-export Reference)", () => { - const names = normalize(fixture).symbols.map(s => s.name); - expect(names).not.toContain("Client"); - }); - - it("does not emit constructor", () => { - const names = normalize(fixture).symbols.map(s => s.name); - expect(names).not.toContain("AuthClient.constructor"); - }); - - it("does not emit private _cache field", () => { - const names = normalize(fixture).symbols.map(s => s.name); - expect(names).not.toContain("AuthClient._cache"); - }); -}); -``` - -- [ ] **Step 2: Run the tests — expect failure** - -```bash -cd scripts/capability-matrix -npx vitest run test/normalize-typedoc.test.ts 2>&1 | head -20 -``` - -Expected: `Cannot find module '../src/normalize-typedoc.js'` - -- [ ] **Step 3: Implement normalize-typedoc.ts** - -Create `scripts/capability-matrix/src/normalize-typedoc.ts`: - -```typescript -export interface ParsedSymbol { - name: string; - kind: "class" | "method" | "property" | "function" | "variable"; - file: string; -} - -export interface ParseResult { - symbols: ParsedSymbol[]; -} - -// TypeDoc ReflectionKind numeric constants used by the normalizer. -const Kind = { - Module: 2, - Namespace: 4, - Enum: 8, - EnumMember: 16, - Variable: 32, - Function: 64, - Class: 128, - Interface: 256, - Constructor: 512, - Property: 1024, - Method: 2048, - Accessor: 262144, - TypeAlias: 2097152, - Reference: 4194304, -} as const; - -interface TdReflection { - name: string; - kind: number; - flags?: { isPrivate?: boolean; isProtected?: boolean }; - sources?: Array<{ fileName: string }>; - children?: TdReflection[]; -} - -function fileOf(r: TdReflection): string { - return r.sources?.[0]?.fileName ?? ""; -} - -function isExcluded(r: TdReflection): boolean { - return !!(r.flags?.isPrivate || r.flags?.isProtected); -} - -function extractMembers( - parent: string, - children: TdReflection[], - out: ParsedSymbol[], -): void { - for (const child of children) { - if (isExcluded(child)) continue; - if (child.kind === Kind.Constructor) continue; - const qualName = `${parent}.${child.name}`; - const file = fileOf(child); - if (child.kind === Kind.Method) { - out.push({ name: qualName, kind: "method", file }); - } else if (child.kind === Kind.Property) { - out.push({ name: qualName, kind: "property", file }); - } else if (child.kind === Kind.Accessor) { - out.push({ name: qualName, kind: "method", file }); - } else if (child.kind === Kind.EnumMember) { - out.push({ name: qualName, kind: "property", file }); - } - } -} - -function extractDeclarations( - children: TdReflection[], - out: ParsedSymbol[], -): void { - for (const child of children) { - if (isExcluded(child)) continue; - const file = fileOf(child); - if (child.kind === Kind.Module || child.kind === Kind.Namespace) { - if (child.children) extractDeclarations(child.children, out); - } else if (child.kind === Kind.Reference) { - continue; - } else if ( - child.kind === Kind.Class || - child.kind === Kind.Interface || - child.kind === Kind.Enum - ) { - out.push({ name: child.name, kind: "class", file }); - if (child.children) extractMembers(child.name, child.children, out); - } else if (child.kind === Kind.Function) { - out.push({ name: child.name, kind: "function", file }); - } else if (child.kind === Kind.Variable || child.kind === Kind.TypeAlias) { - out.push({ name: child.name, kind: "variable", file }); - } - } -} - -export function normalize(json: unknown): ParseResult { - const project = json as TdReflection; - const symbols: ParsedSymbol[] = []; - if (project.children) extractDeclarations(project.children, symbols); - return { symbols }; -} -``` - -- [ ] **Step 4: Update swift-parser.ts to import from normalize-typedoc.ts** - -In `scripts/capability-matrix/src/swift-parser.ts`, find these two lines at the top: - -```typescript -import type { ParsedSymbol, ParseResult } from "./ts-parser.js"; -export type { ParsedSymbol, ParseResult }; -``` - -Change to: - -```typescript -import type { ParsedSymbol, ParseResult } from "./normalize-typedoc.js"; -export type { ParsedSymbol, ParseResult }; -``` - -- [ ] **Step 5: Run the tests — expect all pass** - -```bash -cd scripts/capability-matrix -npx vitest run test/normalize-typedoc.test.ts -``` - -Expected: All tests PASS. If fixture tests fail with "AuthClient not found", re-run Task 1 Step 4 to confirm the fixture shape. - -- [ ] **Step 6: Run the full suite to confirm no regressions** - -```bash -cd scripts/capability-matrix -npm test -``` - -Expected: All tests pass. `ts-parser.test.ts` may fail (it's deleted in Task 5 — that's expected). - -- [ ] **Step 7: Commit** - -```bash -git add scripts/capability-matrix/src/normalize-typedoc.ts \ - scripts/capability-matrix/src/swift-parser.ts \ - scripts/capability-matrix/test/normalize-typedoc.test.ts -git commit -m "feat: add normalize-typedoc with TypeDoc JSON → ParseResult mapping" -``` - ---- - -### Task 3: CLI wrapper and package.json - -**Files:** -- Create: `scripts/capability-matrix/src/normalize-typedoc-cli.ts` -- Modify: `scripts/capability-matrix/package.json` - -**Interfaces:** -- Consumes: `normalize(json: unknown): ParseResult` from `./normalize-typedoc.js` -- Produces: `normalize-typedoc` script; invoked as `npm run normalize-typedoc -- ` - -- [ ] **Step 1: Write the CLI** - -Create `scripts/capability-matrix/src/normalize-typedoc-cli.ts`: - -```typescript -import { readFileSync, writeFileSync } from "node:fs"; -import { normalize } from "./normalize-typedoc.js"; - -const [, , inputPath, outputPath] = process.argv; - -if (!inputPath || !outputPath) { - console.error("Usage: normalize-typedoc "); - process.exit(1); -} - -const json = JSON.parse(readFileSync(inputPath, "utf8")); -const result = normalize(json); -writeFileSync(outputPath, JSON.stringify(result, null, 2)); -``` - -- [ ] **Step 2: Update package.json** - -In `scripts/capability-matrix/package.json`, inside `"scripts"`: - -Replace: -```json -"parse-ts": "tsx src/parse-ts.ts", -``` - -With: -```json -"normalize-typedoc": "tsx src/normalize-typedoc-cli.ts", -``` - -- [ ] **Step 3: Smoke-test the CLI** - -```bash -cd scripts/capability-matrix -npm run normalize-typedoc -- test/fixtures/typedoc-sample.json /tmp/ts-symbols.json -node -e "const r = JSON.parse(require('fs').readFileSync('/tmp/ts-symbols.json','utf8')); console.log(r.symbols.slice(0,5))" -``` - -Expected: Array of 5 `ParsedSymbol` objects with names like `AuthClient`, `AuthClient.signUp`, etc. - -- [ ] **Step 4: Commit** - -```bash -git add scripts/capability-matrix/src/normalize-typedoc-cli.ts \ - scripts/capability-matrix/package.json -git commit -m "feat: add normalize-typedoc-cli and update package.json" -``` - ---- - -### Task 4: Update validate-sdk-compliance.yml - -**Files:** -- Modify: `.github/workflows/validate-sdk-compliance.yml` - -**Interfaces:** -- Produces: TypeScript-conditional steps that replace the `parse-ts` invocation; base/PR symbol files produced at `$GITHUB_WORKSPACE/pr-symbols.json` and `$GITHUB_WORKSPACE/base-symbols.json` (same paths the existing `check-api-symbols` step reads) - -- [ ] **Step 1: Add the entrypoint input** - -In the `workflow_call.inputs` block, add after the existing `sdk-ref` input: - -```yaml - entrypoint: - description: TypeScript entrypoint file relative to SDK root (TypeScript SDKs only) - type: string - default: src/index.ts -``` - -- [ ] **Step 2: Add TypeScript-conditional steps before the existing "Resolve parse command" step** - -In the `check` job, insert the following steps between `Install dependencies` and `Resolve parse command`: - -```yaml - - name: Install PR SDK dependencies (TypeScript) - if: inputs.language == 'javascript' - run: npm ci - working-directory: _sdk-pr - - - name: Generate TypeDoc JSON — PR branch - if: inputs.language == 'javascript' - run: | - npx --yes typedoc@0.27 \ - --json "$GITHUB_WORKSPACE/pr-raw.json" \ - --excludePrivate --excludeProtected \ - ${{ inputs.entrypoint }} - working-directory: _sdk-pr - - - name: Install base SDK dependencies (TypeScript) - if: inputs.language == 'javascript' - run: npm ci - working-directory: _sdk-base - - - name: Generate TypeDoc JSON — base branch - if: inputs.language == 'javascript' - run: | - npx --yes typedoc@0.27 \ - --json "$GITHUB_WORKSPACE/base-raw.json" \ - --excludePrivate --excludeProtected \ - ${{ inputs.entrypoint }} - working-directory: _sdk-base - - - name: Normalize TypeDoc output (TypeScript) - if: inputs.language == 'javascript' - run: | - npm run --silent normalize-typedoc -- \ - "$GITHUB_WORKSPACE/pr-raw.json" "$GITHUB_WORKSPACE/pr-symbols.json" - npm run --silent normalize-typedoc -- \ - "$GITHUB_WORKSPACE/base-raw.json" "$GITHUB_WORKSPACE/base-symbols.json" - working-directory: _sdk-spec/scripts/capability-matrix -``` - -- [ ] **Step 3: Gate the existing parse steps on non-javascript** - -The existing `Resolve parse command`, `Parse PR branch`, and `Parse base branch` steps currently run for all languages. Add `if: inputs.language != 'javascript'` to each: - -```yaml - - name: Resolve parse command - if: inputs.language != 'javascript' - id: resolve - run: | - case "${{ inputs.language }}" in - swift) echo "cmd=parse-swift" >> "$GITHUB_OUTPUT" ;; - *) echo "::error::Unsupported language '${{ inputs.language }}'. Supported values: swift, javascript (javascript uses typedoc path)"; exit 1 ;; - esac - - - name: Parse PR branch - if: inputs.language != 'javascript' - run: | - npm run --silent ${{ steps.resolve.outputs.cmd }} -- "$GITHUB_WORKSPACE/_sdk-pr" \ - > "$GITHUB_WORKSPACE/pr-symbols.json" - working-directory: _sdk-spec/scripts/capability-matrix - - - name: Parse base branch - if: inputs.language != 'javascript' - run: | - npm run --silent ${{ steps.resolve.outputs.cmd }} -- "$GITHUB_WORKSPACE/_sdk-base" \ - > "$GITHUB_WORKSPACE/base-symbols.json" - working-directory: _sdk-spec/scripts/capability-matrix -``` - -The `Check new symbols against capability matrix` step has no `if` condition — it runs for both languages and reads `pr-symbols.json` / `base-symbols.json` regardless of which path produced them. - -- [ ] **Step 4: Validate the YAML** - -```bash -npx js-yaml .github/workflows/validate-sdk-compliance.yml > /dev/null && echo "YAML valid" -``` - -Expected: `YAML valid` - -- [ ] **Step 5: Commit** - -```bash -git add .github/workflows/validate-sdk-compliance.yml -git commit -m "ci: replace parse-ts with typedoc + normalize-typedoc in validate-sdk-compliance" -``` - ---- - -### Task 5: Delete old files and final verification - -**Files:** -- Delete: `scripts/capability-matrix/src/ts-parser.ts` -- Delete: `scripts/capability-matrix/src/parse-ts.ts` -- Delete: `scripts/capability-matrix/test/ts-parser.test.ts` -- Delete: `scripts/capability-matrix/test/fixtures/ts-sample/` - -- [ ] **Step 1: Confirm nothing still imports ts-parser** - -```bash -cd scripts/capability-matrix -grep -r "ts-parser" src/ test/ --include="*.ts" -l -``` - -Expected: No output. If any file appears, update its import to reference `normalize-typedoc.js` instead. - -- [ ] **Step 2: Delete the old files** - -```bash -rm scripts/capability-matrix/src/ts-parser.ts -rm scripts/capability-matrix/src/parse-ts.ts -rm scripts/capability-matrix/test/ts-parser.test.ts -rm -rf scripts/capability-matrix/test/fixtures/ts-sample/ -``` - -- [ ] **Step 3: Run the full test suite** - -```bash -cd scripts/capability-matrix -npm test -``` - -Expected: All tests pass with no failures or type errors. - -- [ ] **Step 4: Run typecheck** - -```bash -cd scripts/capability-matrix -npm run typecheck -``` - -Expected: No errors. If `ts-parser` is still referenced in a type import somewhere, fix the import and re-run. - -- [ ] **Step 5: Commit** - -```bash -git add -u -git commit -m "refactor: remove ts-parser and parse-ts in favor of normalize-typedoc" -``` diff --git a/docs/specs/2026-06-19-ts-parser-typedoc-design.md b/docs/specs/2026-06-19-ts-parser-typedoc-design.md deleted file mode 100644 index 707d5bf..0000000 --- a/docs/specs/2026-06-19-ts-parser-typedoc-design.md +++ /dev/null @@ -1,137 +0,0 @@ -# TypeScript Public API Parser: typedoc Refactor - -**Date:** 2026-06-19 -**Status:** Approved - -## Problem - -`ts-parser.ts` walks individual TypeScript source files using the TypeScript compiler API in syntactic-only mode. It correctly handles top-level `class`, `function`, and `variable` declarations but misses: - -- `interface` and `type` alias declarations -- `enum` declarations and members -- Re-exports (`export * from`, `export { foo } from`) -- Namespace merging and declaration merging -- Symbols only accessible via the package's `exports` field - -This creates false negatives: new public symbols added to the supabase-js SDK may not be caught by the compliance check because the parser cannot see them. - -There is also an architectural inconsistency. PR #35 established the pattern of using an external, well-maintained language tool (dartdoc_json) to generate structured JSON, then normalizing that output into `ParseResult`. The TypeScript path should follow the same shape. - -## Solution - -Replace `ts-parser.ts` and `parse-ts.ts` with a thin normalizer (`normalize-typedoc.ts`) over [TypeDoc](https://typedoc.org/) JSON output. TypeDoc runs the full TypeScript compiler internally and handles all of the cases the current parser misses. The `check-api-symbols` interface and everything downstream are unchanged. - -## End-to-End Flow - -``` -supabase-js PR - → validate-sdk-compliance.yml - [ts-only] npm ci (in SDK repo) - [ts-only] npx typedoc@0.27 --json pr-raw.json --excludePrivate --excludeProtected - [ts-only] git checkout base; npm ci; npx typedoc@0.27 --json base-raw.json … - → normalize-typedoc pr-raw.json → pr-symbols.json - → normalize-typedoc base-raw.json → base-symbols.json - → check-api-symbols pr-symbols.json base-symbols.json sdk-compliance.yaml -``` - -## Files Changed - -### Added -- `scripts/capability-matrix/src/normalize-typedoc.ts` — core normalizer -- `scripts/capability-matrix/src/normalize-typedoc-cli.ts` — CLI entry point -- `scripts/capability-matrix/test/normalize-typedoc.test.ts` — ~15 fixture-based unit tests -- `scripts/capability-matrix/test/fixtures/typedoc-sample.json` — real TypeDoc 0.27 output - -### Deleted -- `scripts/capability-matrix/src/ts-parser.ts` -- `scripts/capability-matrix/src/parse-ts.ts` -- `scripts/capability-matrix/test/ts-parser.test.ts` -- `scripts/capability-matrix/test/fixtures/ts-sample/` - -### Modified -- `.github/workflows/validate-sdk-compliance.yml` — replace `parse-ts` steps with TypeDoc steps; add `entrypoint` input -- `scripts/capability-matrix/package.json` — `normalize-typedoc` script replaces `parse-ts` - -## Normalizer Design - -TypeDoc JSON is a tree of "reflections", each with a numeric `kind`. The normalizer walks two levels: top-level declarations and their members for class-like types. - -### Tree traversal - -With `--entryPointStrategy resolve`, TypeDoc wraps each entrypoint's declarations inside a `Module` reflection (kind 2) rather than placing them directly under the project root. The normalizer flattens one level: it walks `project.children`, and for any child with kind `Module` (2) or `Namespace` (4) it recurses into that child's `children` to find declarations. Namespaces are not emitted as symbols themselves — only their members are. - -### Top-level kind mapping - -| TypeDoc kind | Numeric value | Emitted `ParsedSymbol.kind` | Notes | -|---|---|---|---| -| Class | 128 | `"class"` | walk members | -| Interface | 256 | `"class"` | treated same as class | -| Enum | 8 | `"class"` | walk members | -| Function | 64 | `"function"` | | -| Variable | 32 | `"variable"` | | -| TypeAlias | 2097152 | `"variable"` | exported types worth tracking | -| Reference | 4194304 | skip | re-export pointer; canonical declaration already emitted | - -### Member kind mapping (inside Class / Interface / Enum) - -| TypeDoc kind | Numeric value | Emitted `ParsedSymbol.kind` | -|---|---|---| -| Method | 2048 | `"method"` with `ClassName.name` | -| Property | 1024 | `"property"` with `ClassName.name` | -| Accessor | 262144 | `"method"` (getter/setter) | -| Constructor | 512 | skip | -| EnumMember | 16 | `"property"` with `EnumName.MemberName` | - -### Privacy - -TypeDoc is invoked with `--excludePrivate --excludeProtected`, so private and protected members are stripped before the JSON is written. The normalizer also defensively skips any member where `flags.isPrivate` or `flags.isProtected` is `true`, in case the tool is ever called without those flags. - -### File path - -`ParsedSymbol.file` is populated from `sources[0].fileName` on each reflection (e.g. `src/auth/client.ts`). - -### Normalizer public interface - -```typescript -export function normalize(json: unknown): ParseResult -``` - -A single function. The CLI wrapper reads the input file, calls `normalize`, and writes the output file. - -## CI Workflow Changes - -New `entrypoint` input on `validate-sdk-compliance.yml` (default: `src/index.ts`). SDK repos that have a non-standard entrypoint pass this explicitly; most do not need to change. - -TypeDoc is pinned at `0.27` and invoked via `npx --yes typedoc@0.27` — no changes needed to the SDK's `package.json`. - -The `language: javascript` input value is unchanged; SDK repos that already call this workflow need no modifications. - -## Behavioral Notes - -### `.sdk-parse-ignore` no longer applies to TypeScript - -The old `ts-parser.ts` honored `.sdk-parse-ignore` (gitignore-syntax file) to suppress specific file paths from its output. The TypeDoc path does not use `.sdk-parse-ignore`. TypeDoc naturally limits its output to symbols reachable from the configured entrypoint — internal files and test paths that are not re-exported through the package's `exports` field are not included in the TypeDoc JSON and therefore are not emitted by the normalizer. `.sdk-parse-ignore` continues to apply to Swift parsing via `swift-parser.ts` and `parse-ignore.ts`. - -## Testing - -The fixture (`typedoc-sample.json`) is generated from a minimal TypeScript project that exercises all mapped cases. It is committed to the repo so tests are hermetic and do not require a TypeScript build at test time. - -Test coverage (~15 cases): - -| Case | Assertion | -|---|---| -| Class | emits `"class"` kind | -| Class methods | `ClassName.method` as `"method"` | -| Class property | `ClassName.prop` as `"property"` | -| Accessor | getter/setter as `"method"` | -| Constructor | not emitted | -| Interface | emitted as `"class"` | -| Interface members | same rules as class members | -| Enum | emitted as `"class"` | -| EnumMember | `EnumName.Member` as `"property"` | -| Exported function | `"function"` kind | -| Exported variable | `"variable"` kind | -| TypeAlias | `"variable"` kind | -| Reference (re-export) | not emitted | -| File path | taken from `sources[0].fileName` | -| Defensive privacy | member with `flags.isPrivate: true` not emitted |