Skip to content

ci(workflow): add stats validation workflow and schema checks - #432

Closed
vjymisal0 wants to merge 1 commit into
e18e:mainfrom
vjymisal0:feat/validation-ci-check
Closed

ci(workflow): add stats validation workflow and schema checks#432
vjymisal0 wants to merge 1 commit into
e18e:mainfrom
vjymisal0:feat/validation-ci-check

Conversation

@vjymisal0

Copy link
Copy Markdown

Fixes #431.

Summary of Changes

  1. Validation Workflow (.github/workflows/validate-stats-pr.yml):

    • Triggers on pull requests updating stats files (packages/starter-*/ci-stats.json, packages/app-*/ci-stats.json, packages/docs/src/content/**).
    • Runs the test suite and executes validate:json across all modified / existing stats JSON files.
  2. Zod Validation Schemas (packages/stats-generator/src/schemas.ts):

    • Added StarterCIStatsSchema and AppCIStatsSchema to enforce presence and validity of required stats properties (install times, build times, output size, module sizes, dependency counts, and benchmark measurements).
    • Enforces that benchmark measurements contain positive numeric metrics and cannot be set to 0 or missing.
  3. JSON Stats Validator (packages/stats-generator/src/validate-json-stats.ts):

    • Recursively inspects all starter packages, app packages, versioned history files, and docs content.
    • Validates each file against its schema and provides clear error reporting with non-zero exit codes upon schema violations.
    • Added validate:json script in packages/stats-generator/package.json.
  4. Unit Tests:

    • Added packages/stats-generator/src/validate-json-stats.test.ts to test schema verification for starter and app payloads.

@AlexanderKaran

AlexanderKaran commented Aug 29, 2026

Copy link
Copy Markdown
Collaborator

Hey @vjymisal0

So this is going to run on more than just the CI Stats PR, which is not what we want.

Also, you're creating brand new schemas when we already have some we can reuse.

We're currently adding versioning to the schema, and I wasn't ready for someone to jump on this issue straight away.

I am going to take it down for now and close this PR.

If you would like to get involved, let me know, as I have some other areas you can work on :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Validation CI Check

2 participants