Skip to content

Add agentic workflow for automated issue triage#4177

Open
priyankatiwari08 wants to merge 7 commits intodotnet:mainfrom
priyankatiwari08:feature/agentic-issue-triage
Open

Add agentic workflow for automated issue triage#4177
priyankatiwari08 wants to merge 7 commits intodotnet:mainfrom
priyankatiwari08:feature/agentic-issue-triage

Conversation

@priyankatiwari08
Copy link
Copy Markdown
Contributor

@priyankatiwari08 priyankatiwari08 commented Apr 10, 2026

Summary

This PR introduces a GitHub Agentic Workflow (gh-aw) that automatically triages new issues using Copilot.

What it does

When a new issue is opened, the workflow:

  1. Classifies issue type — Bug, Feature, Question, or Task
  2. Validates environment details (bugs only) — checks for SqlClient version, .NET target framework, SQL Server version, OS, repro steps, expected/actual behavior
  3. Assigns area labels — matches issue content against 14 area labels (e.g. Area\Async, Area\Connection Pooling, Area\Managed SNI)
  4. Searches for duplicates — searches repo:dotnet/SqlClient for similar issues
  5. Checks for regressions — if the reporter mentions a previously working version
  6. Posts a single consolidated triage summary — one comment with a structured table (issue type, environment, area, duplicates, regression) + analysis + next steps
  7. Assigns Copilot coding agent — for confirmed bugs with complete environment info

Files added

1. .github/workflows/issue-triage.md — Workflow Definition

This is the only file that needs to be maintained. It contains:

  • YAML frontmatter: Defines the trigger (issues: opened), the AI engine (copilot), and safety limits (max 1 comment, max 5 labels).
  • Markdown body: Natural language instructions that tell the Copilot agent how to triage each issue — role definition, analysis steps (classify type, validate environment, match area labels, search duplicates, check regression), area label lookup table, comment template, and action sequence.

This file is written in plain Markdown because GitHub Agentic Workflows use an LLM (Copilot) to interpret instructions, unlike traditional YAML workflows that use scripted steps. When changes to the triage logic are needed (e.g. adding a new area label, changing the comment format), only this file is edited, then recompiled.

2. .github/workflows/issue-triage.lock.yml — Compiled Actions YAML (~61KB)

This is the auto-generated GitHub Actions workflow file produced by running gh aw compile issue-triage.md. It is what GitHub Actions actually executes when triggered. It contains:

  • The full two-job Actions YAML infrastructure (agentic job + safe-output processor job)
  • The instructions from issue-triage.md embedded as runtime imports
  • Safe-outputs enforcement configuration (max limits, allowed actions)
  • MCP server setup (read-only GitHub MCP + write-sink safe-outputs MCP)

This file should never be edited manually. It is regenerated every time issue-triage.md is compiled. The edit-compile-push cycle is: edit .md → run gh aw compile → commit both files → push.

3. .github/aw/actions-lock.json — Pinned Action Versions

This is a lockfile generated by gh aw compile that pins the exact versions of the GitHub Actions used by the agentic workflow runtime. Similar to package-lock.json in Node.js or Cargo.lock in Rust, it ensures the workflow uses the same action versions on every run, preventing unexpected behavior from upstream action updates.

This file should never be edited manually. It is regenerated automatically during compilation.

Architecture

The workflow uses the safe-outputs pattern:

  • Job 1 (Agentic): Copilot agent reads the issue via read-only GitHub MCP, reasons about it, and proposes actions (labels, comment, agent assignment) written to NDJSON files
  • Job 2 (Safe-output Processor): A separate privileged job validates proposed actions against configured limits (max 1 comment, max 5 labels) and executes them

This ensures the AI agent never has direct write access to the repository.

Safety limits configured

  • add-comment: max: 1 — at most 1 comment per issue
  • add-labels: max: 5 — at most 5 labels per issue
  • hide-older-comments: true — collapses previous triage comments if re-triggered

Prerequisites (post-merge)

The following setup is required by a repo admin after merging:

  1. Set agentic workflow secret:

    gh aw secrets set COPILOT_GITHUB_TOKEN --value <token>
    
  2. Set agent assignment secret (required — enables automatic Copilot coding agent assignment for confirmed bugs with complete environment details):

    gh aw secrets set GH_AW_AGENT_TOKEN --value <token>
    
  3. Create missing labels (these don't exist yet on this repo):

    • Triage Needed :new: — applied to all new issues
    • Needs More Info :information_source: — applied when environment details are missing
  4. Ensure gh-aw is enabled for the dotnet org / SqlClient repo

Tested on

This workflow was developed and tested on priyankatiwari08/SqlClient-test-prtiwar with multiple test issues covering:

  • Complete bug reports (all env details present) — correctly triaged with area labels + Copilot assignment
  • Incomplete bug reports (missing OS, .NET TF, SQL Server version) — correctly flagged missing fields + Needs More Info label
  • Various area classifications (Async, Managed SNI, Sql Bulk Copy) — correctly matched

Introduces a GitHub Agentic Workflow (gh-aw) that automatically triages
new issues using Copilot. The workflow:

- Classifies issue type (Bug/Feature/Question/Task)
- Validates environment details for bugs
- Assigns area labels based on issue content
- Searches for duplicate issues in the repo
- Checks for regressions
- Posts a single consolidated triage summary comment
- Assigns Copilot coding agent for confirmed bugs with complete info

Files added:
- .github/workflows/issue-triage.md (workflow definition)
- .github/workflows/issue-triage.lock.yml (compiled Actions YAML)
- .github/aw/actions-lock.json (pinned action versions)
@github-project-automation github-project-automation Bot moved this to To triage in SqlClient Board Apr 10, 2026
Copilot AI review requested due to automatic review settings April 10, 2026 08:10
@priyankatiwari08 priyankatiwari08 requested a review from a team as a code owner April 10, 2026 08:10
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds a GitHub Agentic Workflow (gh-aw) for automated issue triage in the SqlClient repo, using a Copilot-powered agent plus a privileged “safe-outputs” executor job.

Changes:

  • Added an agent-authored workflow definition (issue-triage.md) describing triage logic and safe-output constraints.
  • Added the compiled/locked GitHub Actions workflow (issue-triage.lock.yml) generated by gh aw compile.
  • Added an action lockfile (.github/aw/actions-lock.json) pinning action SHAs.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 7 comments.

File Description
.github/workflows/issue-triage.md Defines the agent prompt, triage steps, and safe-output configuration for labeling/commenting/assignment.
.github/workflows/issue-triage.lock.yml Auto-generated compiled Actions workflow that executes the agent + safe-outputs pipeline.
.github/aw/actions-lock.json Pins action versions used by the compiled workflow.

Comment thread .github/workflows/issue-triage.md
Comment thread .github/workflows/issue-triage.md
Comment thread .github/workflows/issue-triage.md Outdated
Comment thread .github/workflows/issue-triage.md Outdated
Comment thread .github/workflows/issue-triage.lock.yml Outdated
Comment thread .github/workflows/issue-triage.lock.yml
Comment thread .github/workflows/issue-triage.md Outdated
Copilot AI review requested due to automatic review settings April 10, 2026 08:50
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.

Comment thread .github/workflows/issue-triage.md Outdated
Comment thread .github/workflows/issue-triage.md Outdated
Comment thread .github/workflows/issue-triage.lock.yml Outdated
Comment thread .github/aw/actions-lock.json
…s, fix Bug classification wording, fix Needs More Info label format
@paulmedynski paulmedynski moved this from To triage to In review in SqlClient Board Apr 10, 2026
Copilot AI review requested due to automatic review settings April 13, 2026 11:05
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (1)

.github/workflows/issue-triage.md:130

  • This file says the workflow always reads the issue, applies labels, and posts one summary comment, but later suggests calling noop when “no action is needed”. As written, it’s ambiguous whether noop should replace the labeling/comment steps or be in addition to them, which could lead to either missing triage output or emitting conflicting safe-outputs. Please clarify the control flow (e.g., if noop is used, explicitly say to skip add_labels/add_comment, or remove noop if a comment is always required).
**Finally**: If this is a confirmed code bug with complete environment info,
call `assign_to_agent` to assign Copilot coding agent.

If the issue is spam or no action is needed, call the `noop` tool instead.

Comment thread .github/workflows/issue-triage.md
Comment thread .github/workflows/issue-triage.md Outdated
Comment thread .github/workflows/issue-triage.md
Comment thread .github/workflows/issue-triage.lock.yml
Comment thread .github/workflows/issue-triage.md
@github-project-automation github-project-automation Bot moved this from In review to In progress in SqlClient Board Apr 15, 2026
Copilot AI review requested due to automatic review settings April 23, 2026 19:44
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

Comment on lines +1088 to +1090
discussions: write
issues: write
pull-requests: write
Copy link

Copilot AI Apr 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The generated safe_outputs job grants discussions: write and pull-requests: write, but this workflow only triggers on issues: opened and (per the .md instructions) only needs to add issue labels/comments. If possible, reduce these job permissions to least-privilege (e.g., issues: write + contents: read) or document why broader write scopes are required by gh-aw for this workflow.

Suggested change
discussions: write
issues: write
pull-requests: write
issues: write

Copilot uses AI. Check for mistakes.
Comment on lines +47 to +53
Before analyzing the issue, you MUST read all project knowledge base files
from the checked-out repository. Recursively list the `.github/` directory
and read every markdown file (`.md`) found under it, excluding the `workflows/`
subdirectory. This includes but is not limited to instructions, prompts,
issue templates, skills, plans, and any other documentation files present.

Use these files to inform your area classification, duplicate detection,
Copy link

Copilot AI Apr 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The workflow requires reading every Markdown file under .github/ (excluding workflows/) before triaging. In this repo that includes many large instruction/prompt docs, which is likely to significantly increase runtime/token usage per issue and may exceed the model context window—causing inconsistent triage. Consider narrowing this to a small, explicit allowlist of the most relevant knowledge sources (e.g., issue templates + copilot-instructions.md + a specific triage prompt), or make the full scan conditional/fallback when classification is ambiguous.

Suggested change
Before analyzing the issue, you MUST read all project knowledge base files
from the checked-out repository. Recursively list the `.github/` directory
and read every markdown file (`.md`) found under it, excluding the `workflows/`
subdirectory. This includes but is not limited to instructions, prompts,
issue templates, skills, plans, and any other documentation files present.
Use these files to inform your area classification, duplicate detection,
Before analyzing the issue, read only the core project knowledge sources
most relevant to triage from the checked-out repository:
- `.github/copilot-instructions.md`
- files under `.github/ISSUE_TEMPLATE/`
- `.github/prompts/issue-triage.prompt.md` if that file exists
Do NOT recursively read every markdown file under `.github/`.
Exclude `.github/workflows/`.
If issue classification, required-field validation, or area detection is still
ambiguous after reading the allowlisted files above, you may read a small number
of additional directly relevant markdown files under `.github/` (excluding
`workflows/`) that are specifically related to the ambiguity you need to resolve.
Do not perform a full-directory markdown scan.
Use the files you read to inform your area classification, duplicate detection,

Copilot uses AI. Check for mistakes.
Comment on lines +23 to +27
max: 5
# Phase 2: Uncomment when ready to enable automatic Copilot coding agent assignment
# assign-to-agent:
# github-token: ${{ secrets.GH_AW_AGENT_TOKEN }}
---
Copy link

Copilot AI Apr 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR description says the workflow will automatically assign the Copilot coding agent for confirmed bugs, but assign-to-agent is currently commented out in safe-outputs (and the compiled issue-triage.lock.yml contains no assignment support). Either enable the assignment safe-output/tools in this PR, or update the PR description/prerequisites to reflect that agent assignment is not yet active.

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: In progress

Development

Successfully merging this pull request may close these issues.

4 participants