Add the code testing agent and /add-tests workflow

[JanKrivanek]

Motivation

Add the code testing agent and it's hooking into slash commend
This should help driving the test coverage for the changes added by PRs

Adapted from: dotnet/machinelearning#7602

cc: @Evangelink

[Copilot]

Pull request overview

Adds a new /add-tests slash-command workflow (via gh-aw) intended to generate unit tests for PR changes, and introduces a reusable “code testing agent” skill + agent definitions to drive the generation pipeline.

Changes:

• Added /add-tests workflow source (add-tests.md), shared repo build setup, and compiled lock workflow (add-tests.lock.yml).
• Introduced a new code-testing-agent skill (prompt + .NET extension guidance).
• Added a set of code-testing-* agent definitions (research/plan/implement/build/test/fix/lint orchestration).

Show a summary per file

File	Description
.github/workflows/shared/repo-build-setup.md	Repo-specific build steps shared by the add-tests workflow.
.github/workflows/add-tests.md	gh-aw workflow manifest/prompt that defines the /add-tests behavior.
.github/workflows/add-tests.lock.yml	Compiled GitHub Actions workflow that executes the agentic flow.
.github/skills/code-testing-agent/unit-test-generation.prompt.md	Baseline guidance for generating high-quality unit tests (80% target).
.github/skills/code-testing-agent/SKILL.md	Skill wrapper describing the Research→Plan→Implement pipeline and state files.
.github/skills/code-testing-agent/extensions/dotnet.md	.NET-specific build/test/lint guidance for test generation.
.github/aw/actions-lock.json	Locks SHAs for selected actions used by the workflow framework.
.github/agents/code-testing-generator.agent.md	Orchestrator agent for the RPI test-generation pipeline.
.github/agents/code-testing-researcher.agent.md	Research sub-agent to map codebase/tests/commands.
.github/agents/code-testing-planner.agent.md	Planning sub-agent to produce phased test plan.
.github/agents/code-testing-implementer.agent.md	Implementation sub-agent to write tests and verify build/test.
.github/agents/code-testing-builder.agent.md	Build sub-agent guidance.
.github/agents/code-testing-tester.agent.md	Test execution sub-agent guidance.
.github/agents/code-testing-fixer.agent.md	Compilation error fixing sub-agent guidance.
.github/agents/code-testing-linter.agent.md	Formatting/linting sub-agent guidance.

Copilot's findings

Comments suppressed due to low confidence (1)

.github/workflows/add-tests.md:72

• This section tells the agent to use the task tool to invoke code-testing-generator, but the workflow doesn’t grant that tool (see tools: allowlist and compiled --allow-tool list). Update either the tool allowlist or the instructions so the workflow isn’t asking the agent to do something it cannot do.

### Step 3: Generate Tests

Use the `code-testing-generator` agent (defined at `.github/agents/code-testing-generator.agent.md`) via the task tool to generate tests:

1. Follow the Research → Plan → Implement pipeline from the skill

• Files reviewed: 15/15 changed files
• Comments generated: 4

[Youssef1313]

This is the same from dotnet/skills right?

I'm adding it already in #7707 and did automation for it.

It was pending markdownlint violations that I fixed in dotnet/skills#513

[JanKrivanek]

Thanks @Youssef1313! Yes, the agent/skill files are the same ones from dotnet/skills - I've rebased the branch on main so this PR now only contains the workflow files (add-tests.md, add-tests.lock.yml, repo-build-setup.md, actions-lock.json). The agent/skill definitions come from your #7711.

[JanKrivanek]

Thanks @Youssef1313! The agent/skill files are now coming from your #7711 - I rebased the branch on main so this PR only adds the workflow files (add-tests.md, add-tests.lock.yml, repo-build-setup.md, actions-lock.json). Also addressed your feedback about preferring integration tests over unit tests for MSTest framework code.

[Copilot]

Pull request overview

Adds an agentic /add-tests GitHub workflow to generate tests for PR changes, including repo-specific build setup and workflow/action lock artifacts to support running in this repository.

Changes:

• Introduces the /add-tests slash-command workflow definition (add-tests.md) with build setup import and safe-output constraints for creating a draft PR with generated tests.
• Adds repo-specific build setup steps shared by the workflow (shared/repo-build-setup.md).
• Adds compiled/locked workflow output (add-tests.lock.yml) and an actions lock metadata file (.github/aw/actions-lock.json) for reproducibility/auditing.

Show a summary per file

File	Description
.github/workflows/shared/repo-build-setup.md	Defines repo-specific build + dotnet PATH setup imported by the workflow.
.github/workflows/add-tests.md	Declares the /add-tests workflow trigger, tool allowlist, safe outputs, and the agent instructions.
.github/workflows/add-tests.lock.yml	Auto-generated compiled workflow used by GitHub Actions to run the agentic pipeline.
.github/aw/actions-lock.json	Records pinned action SHAs intended to support reproducible workflow execution.

Copilot's findings

Comments suppressed due to low confidence (3)

.github/workflows/add-tests.md:64

• The test-project mapping for src/Analyzers/MSTest.Analyzers/ points to test/UnitTests/MSTest.Analyzers.Tests/, but this repo uses test/UnitTests/MSTest.Analyzers.UnitTests/ (no MSTest.Analyzers.Tests directory exists). Update the mapping so the workflow generates tests in the correct project.

   - `src/Adapter/MSTest.Engine/` → `test/UnitTests/MSTest.Engine.UnitTests/`
   - `src/Analyzers/MSTest.Analyzers/` → `test/UnitTests/MSTest.Analyzers.Tests/` (if exists)
   - `src/Analyzers/MSTest.SourceGeneration/` → `test/UnitTests/MSTest.SourceGeneration.UnitTests/`
   - `src/Platform/` → `test/UnitTests/` (find matching test project by name)

.github/workflows/add-tests.md:77

• This workflow instruction mandates FluentAssertions for all new tests, but several existing test projects in this repo explicitly don’t use FluentAssertions (e.g., MSTest.Engine.UnitTests sets <NoFluentAssertions>true</NoFluentAssertions>, analyzer tests use verifier helpers). Consider changing the guidance to “follow the assertion style used by the target test project; use FluentAssertions where available.”

2. **Scope**: Only generate tests for code modified in this PR — do not attempt full-repo coverage
3. **Test framework**: This repo uses MSTest with `[TestMethod]`, `[DataRow]` attributes for MTP and analyzer tests. Unit tests for MSTest itself MUST use the internal test framework from `test/Utilities/TestFramework.ForTestingMSTest`
4. **Test type preference**: For MSTest framework code, **prefer integration tests** (under `test/IntegrationTests/`) over unit tests — unit tests are often not sufficient to validate framework behavior. The exception is `MSTest.Analyzers`, where unit tests are appropriate
5. **Assertions**: All assertions must use FluentAssertions style
6. **Naming**: Test classes as `{Feature}Tests`, test methods as PascalCase descriptive names following `Method_Condition_ExpectedResult` pattern

.github/workflows/add-tests.md:72

• The prompt instructs the agent to invoke code-testing-generator “via the task tool”, but this workflow’s declared tool allowlist only includes github, edit, and bash. If the runtime doesn’t expose a task/subagent capability, the agent won’t be able to follow these instructions. Either add the needed tool(s) (if supported by gh-aw/Copilot CLI) or reword Step 3 to follow the R→P→I pipeline directly without requiring the task tool.

### Step 3: Generate Tests

Use the `code-testing-generator` agent (defined at `.github/agents/code-testing-generator.agent.md`) via the task tool to generate tests:

1. Follow the Research → Plan → Implement pipeline from the skill

• Files reviewed: 4/4 changed files
• Comments generated: 3

[Copilot]

Pull request overview

Introduces a new /add-tests slash-command workflow (gh-aw compiled) that runs a Copilot-driven agent to generate tests for PR changes, using a repo-specific build setup component.

Changes:

• Added add-tests gh-aw workflow prompt (.md) plus repo build setup import.
• Added compiled/locked GitHub Actions workflow (add-tests.lock.yml) for execution in CI.
• Added gh-aw action lock metadata (.github/aw/actions-lock.json).

Show a summary per file

File	Description
.github/workflows/shared/repo-build-setup.md	Defines repo-specific build steps for the add-tests workflow (build + add local dotnet to PATH).
.github/workflows/add-tests.md	Workflow prompt/instructions for the agent to locate changed src/ code and generate appropriate tests.
.github/workflows/add-tests.lock.yml	Compiled gh-aw workflow that triggers on /add-tests PR comments and runs the agent with safe-outputs.
.github/aw/actions-lock.json	Records pinned action SHAs used by gh-aw (lock metadata).

Copilot's findings

• Files reviewed: 4/4 changed files
• Comments generated: 4

[Copilot]

src/Analyzers/MSTest.Analyzers/ is mapped to test/UnitTests/MSTest.Analyzers.Tests/ here, but that test project path doesn’t exist in this repo (the current project is test/UnitTests/MSTest.Analyzers.UnitTests/). Updating this mapping will help the /add-tests agent reliably locate the right test project and avoid generating tests into a non-existent folder.

Suggested change

	- `src/Analyzers/MSTest.Analyzers/` → `test/UnitTests/MSTest.Analyzers.Tests/` (if exists)
	- `src/Analyzers/MSTest.Analyzers/` → `test/UnitTests/MSTest.Analyzers.UnitTests/`

[Copilot]

The repo’s tests use the AwesomeAssertions package (with FluentAssertions-style Should() syntax), not FluentAssertions itself. If the instructions say “use FluentAssertions” without clarifying, the agent may generate using FluentAssertions; which won’t compile. Consider updating this guidance to explicitly use AwesomeAssertions (or at least clarify that the expected library in this repo is AwesomeAssertions with FluentAssertions-style assertions).

Suggested change

	5. **Assertions**: All assertions must use FluentAssertions style
	5. **Assertions**: All assertions must use `AwesomeAssertions` with FluentAssertions-style `Should()` syntax; do not use or import `FluentAssertions`

[Copilot]

Consider quoting $GITHUB_PATH when appending to it (e.g., >> "$GITHUB_PATH"). While it’s typically safe on GitHub-hosted runners, quoting avoids edge cases with unusual path values and matches common GitHub Actions patterns.

Suggested change

	run: echo "$PWD/.dotnet" >> $GITHUB_PATH
	run: echo "$PWD/.dotnet" >> "$GITHUB_PATH"

[Copilot]

permissions: {} at the workflow level disables all GITHUB_TOKEN permissions by default, but the pre_activation job doesn’t override permissions:. Since pre_activation runs actions/github-script (e.g., to check membership), it’s likely to fail with 403 “Resource not accessible by integration”. Add an explicit permissions: block to pre_activation with the minimal required scopes (or adjust the gh-aw generation so pre_activation gets the needed permissions).

Suggested change

	runs-on: ubuntu-slim
	runs-on: ubuntu-slim
	permissions:
	contents: read