Skip to content

contributor-activity-sweep skill with eval suite#369

Open
justinmclean wants to merge 4 commits into
apache:mainfrom
justinmclean:contributor-activity-sweep
Open

contributor-activity-sweep skill with eval suite#369
justinmclean wants to merge 4 commits into
apache:mainfrom
justinmclean:contributor-activity-sweep

Conversation

@justinmclean
Copy link
Copy Markdown
Member

@justinmclean justinmclean commented May 29, 2026

Summary

Rebased continuation of #228

Type of change

  • Skill change (.claude/skills/<name>/) — eval fixtures updated below
  • Tool / bridge contract (tools/<system>/*.md)
  • Python package (tools/*/ with pyproject.toml)
  • Groovy reference impl
  • Cross-cutting (RFC, AGENTS.md, sandbox, privacy-LLM)
  • Documentation (docs/, README.md, CONTRIBUTING.md)
  • Project template (projects/_template/)
  • CI / dev loop (prek, workflows, validators)
  • Other:

Test plan

  • prek run --all-files passes
  • For Python packages touched: uv run pytest / ruff check / mypy passes
  • For Groovy bridges touched: command-line invocation tested end-to-end
  • For skill changes: eval suite passes for the affected skill
    (PYTHONPATH=tools/skill-evals/src python3 -m skill_evals.runner tools/skill-evals/evals/<skill>/)
  • For skill behaviour changes: a new or updated eval fixture is included in this PR
    (a regression test for the bug fixed / the behaviour added — see CONTRIBUTING.md)
  • Other:

Self-review fixes for the two advisory findings:

- Trim the frontmatter description: drop the duplicate enum of off-GitHub
  channels (it is already documented prominently in the body callout).
  Comma count drops from 5+ to 3, clearing the skill-validator --strict
  action-inventory soft warning. The matching-layer wording is otherwise
  unchanged.
- Rename "## Step 1 — Fetch GitHub activity" to "## Step 1 — Fetch and
  classify activity" so the heading matches the eval dir
  (step-1-classify-reviews) and reflects what Step 1 actually does
  (fetch + substantive/LGTM classification). Update the matching
  step-config.json step_heading accordingly; the eval renderer
  continues to extract the section correctly.

Generated-by: Claude Code (Opus 4.7)
@justinmclean justinmclean force-pushed the contributor-activity-sweep branch from c77c040 to bdbd894 Compare May 29, 2026 02:52
@justinmclean justinmclean self-assigned this May 29, 2026
@justinmclean justinmclean marked this pull request as ready for review June 1, 2026 01:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants