[copilot-cli-research] Copilot CLI Deep Research - 2026-04-29

[github-actions[bot]]

Analysis Date: 2026-04-29
Repository: github/gh-aw
Scope: 205 total workflows, 110 using Copilot engine (54%), 47 Claude, 10 Codex

---

📊 Executive Summary

Research Topic: Copilot CLI Feature Usage & Optimization Opportunities
Key Findings: startup-timeout is NEVER used (0%, 11th consecutive run), max-continuations remains near-zero (2 workflows), web-search underutilized (2 workflows), sandbox AWF coverage is low (15%), and toolsets scoping is inconsistent.
Primary Recommendation: Adopt startup-timeout in long-running and read-heavy workflows — it costs nothing and protects against hung sessions.

This is the 6th consecutive deep-research run on this repository. The workflow count continues to grow (203 → 205), with Copilot engine adoption stable at ~54%. Most persistent gaps from prior runs remain unchanged. On the positive side, cache-memory adoption is healthy (79 workflows, 72%), and repo-memory usage has grown to 23 workflows (21%).

The most actionable short-term win remains startup-timeout — it has been at 0% for 11 consecutive analysis runs, yet it protects every single workflow from silently hanging during Copilot CLI initialization.

---

Critical Findings

🔴 High Priority Issues

1. startup-timeout unused across ALL 110 Copilot workflows (0%, 11th consecutive run)
   Copilot CLI can silently hang during startup. Without a startup timeout, workflows consume runner minutes until the step-level timeout-minutes fires — potentially 60+ minutes later. Adding startup-timeout: 120 (2 min) is a free safety net.

2. Sandbox (AWF) enabled in only 17/110 Copilot workflows (15%)
   The AWF network firewall prevents data exfiltration and prompt-injection network calls. Workflows with edit: or broad bash: access that lack sandbox: agent: awf are running without this protection.

3. max-continuations used in only 2/110 workflows despite autopilot being supported
   Most multi-step workflows (daily reports, code fixers, refactoring agents) would benefit from max-continuations: 3-10 to allow the agent to self-correct and continue complex tasks.

🟡 Medium Priority Opportunities

4. web-search used in only 2 workflows — available as a builtin tool but almost unused. Useful for research-heavy workflows.
5. mcp-scripts used in only 6 workflows — powerful for custom tooling but low adoption.
6. Inconsistent GitHub toolset scoping — some workflows use toolsets: [default] (over-permissioned), others use no scoping.
7. 0 workflows pin a Copilot CLI engine version — critical workflows might benefit from version pinning.

---

View Full Analysis

1️⃣ Copilot CLI Capabilities Inventory

View Available Features

Core Flags (always applied by compiler)

• --add-dir /tmp/gh-aw/ and --add-dir "${GITHUB_WORKSPACE}" (workspace access)
• --log-level all, --log-dir <path> (full logging)
• --disable-builtin-mcps (always applied; MCP servers go through gateway)
• --no-ask-user (non-interactive mode, v1.0.19+)

Configurable via Frontmatter

Feature	Frontmatter Key	Notes
Custom agent file	engine.agent	Points to a .github/copilot-agents/*.yaml agent
Model selection	engine.model	e.g., gpt-5-mini, claude-haiku-4.5
Version pin	engine.version	Pin specific Copilot CLI version
Bare mode	engine.bare / bare: true	Disables .github/copilot-instructions.md
Max continuations	max-continuations	Enables --autopilot --max-autopilot-continues N
Startup timeout	startup-timeout	Kills if CLI doesn't start within N seconds
Tool timeout	tool-timeout	Per-tool execution timeout
Network restrictions	network.allowed	Allowlist of domains/presets
Sandbox (AWF)	sandbox.agent: awf	Network firewall via AWF
Sandbox (SRT)	sandbox.agent: srt	Stricter sandbox environment
Cache memory	tools.cache-memory	Persistent storage across runs
Repo memory	tools.repo-memory	Git-branch-backed persistent memory
MCP scripts	mcp-scripts	Custom MCP tool scripts
Web fetch	tools.web-fetch	Builtin HTTP fetch tool
Web search	tools.web-search	Builtin web search (rare use)
GitHub MCP	tools.github	GitHub API access via MCP

View Usage Statistics

Feature Adoption Across 110 Copilot Workflows

Feature	Count	Rate	Trend
cache-memory	79	72%	✅ Growing
strict mode	~85	77%	✅ High
safe-outputs	~80	73%	✅ High
imports	~65	59%	✅ Good
repo-memory	23	21%	📈 Growing
web-fetch	19	17%	→ Stable
sandbox (AWF)	17	15%	→ Stable
github toolsets	25	23%	→ Stable
engine.agent	22	20%	📈 Growing
engine.model	10	9%	→ Stable
bare mode	8	7%	→ Stable
mcp-scripts	6	5%	→ Stable
web-search	2	2%	→ Stagnant
max-continuations	2	2%	→ Stagnant
startup-timeout	0	0%	🔴 Persistent gap (11 runs)
tool-timeout	0	0%	🔴 Persistent gap
engine.version	0	0%	🔴 Persistent gap
network restrictions	12	11%	→ Low

---

2️⃣ Missed Opportunities

View High Priority Opportunities

🔴 Opportunity 1: Add startup-timeout to all long-running workflows

What: The startup-timeout config controls how long gh-aw waits for Copilot CLI to start accepting prompts. Currently 0/110 workflows set this.

Why It Matters: If Copilot CLI hangs during initialization (network timeout, token issue, rate limit), the workflow burns runner minutes until timeout-minutes fires — potentially 60+ minutes later.

How to Implement:

engine: copilot
startup-timeout: 120  # Kill if CLI doesn't start within 2 minutes

Affected workflows: All 110 Copilot workflows — but especially daily/weekly reports, code fixers, and analysis workflows.

---

🔴 Opportunity 2: Enable AWF sandbox on write-capable workflows

What: 17/110 Copilot workflows use sandbox: agent: awf. The remaining 93 run without a network firewall.

Why It Matters: Workflows with edit: or broad bash: access can be manipulated via prompt injection to exfiltrate data. AWF blocks outbound network to non-allowlisted domains.

How to Implement:

sandbox:
  agent: awf
network:
  allowed:
    - defaults
    - github

View Medium Priority Opportunities

🟡 Opportunity 3: Use max-continuations for complex iterative agents

What: Only 2 workflows use max-continuations (values: 2 and 40). This enables --autopilot --max-autopilot-continues N, allowing the agent to continue automatically.

Why It Matters: Workflows like dead-code-remover.md, refactoring-cadence.md, daily-workflow-updater.md perform iterative multi-step tasks but run as single-continuation sessions.

engine: copilot
max-continuations: 5  # Allow 5 autopilot continuations

---

🟡 Opportunity 4: Adopt web-search for research-oriented workflows

What: Only 2 workflows use the web-search builtin. Workflows like research.md, weekly-blog-post-writer.md, daily-news.md are research-heavy but don't use web search.

tools:
  web-search:

---

🟡 Opportunity 5: Scope GitHub toolsets more precisely

Some workflows use toolsets: [default] (broad access). Recommendations:

• Issue-only workflows → toolsets: [issues]
• PR review → toolsets: [pull_requests]
• Code scanning → toolsets: [code_security]
• Mixed analysis → toolsets: [default] is appropriate

---

🟡 Opportunity 6: Activate unused custom agent files

These .github/copilot-agents/ files are defined but no workflow uses them:

• grumpy-reviewer — Critical PR review persona
• w3c-specification-writer — Spec writing persona
• create-safe-output-type — Safe output tooling
• custom-engine-implementation — Engine development
• interactive-agent-designer — Agent design tool

Each can be activated with:

engine:
  id: copilot
  agent: grumpy-reviewer

View Low Priority Opportunities

🟢 Opportunity 7: Version pinning for critical workflows

0 workflows pin engine.version. For critical automated workflows like release.md or daily-secrets-analysis.md, pinning can prevent breakage from CLI updates.

engine:
  id: copilot
  version: v1.0.19

🟢 Opportunity 8: Use tool-timeout for bash-heavy workflows

0 workflows use tool-timeout. Bash tools in super-linter.md or dead-code-remover.md can run arbitrarily long. A 300s tool timeout prevents one slow tool call from blocking the entire session.

tool-timeout: 300

---

3️⃣ Trends vs Previous Analysis (2026-04-28)

Metric	Previous	Current	Change
Total workflows	203	205	+2
Copilot workflows	89	110	+21 (method improvement)
cache-memory	29	79	+50 (all forms counted)
repo-memory	N/A	23	New metric
sandbox AWF	11	17	+6 ✅
web-fetch	7	19	+12 ✅
mcp-scripts	0	6	+6 ✅
engine.agent	0	22	+22 ✅
max-continuations	2	2	→ unchanged
startup-timeout	0	0	🔴 11th run at 0
tool-timeout	0	0	🔴 persistent

---

4️⃣ Best Practice Guidelines

1. Always set startup-timeout: 120 — Free protection against hung sessions.
2. Use AWF sandbox for write-capable workflows — Any workflow with edit: or bash: ["*"] should have sandbox: agent: awf.
3. Scope GitHub toolsets to what you need — Prefer [issues], [pull_requests], [repos] over always [default].
4. Use max-continuations: 3-10 for iterative tasks — Code fixers, refactoring agents, and report generators all benefit.
5. Use repo-memory for trend analysis — 23 workflows already use it; great for recurring reports.
6. Add web-search for research workflows — Low friction, high value for news/research/competitive analysis.

---

7️⃣ Action Items

Immediate Actions (this week):

• Add startup-timeout: 120 to the top 10 highest-impact Copilot workflows
• Enable sandbox: agent: awf on write-capable workflows missing it

Short-term (this month):

• Activate at least 2 unused custom agent files (grumpy-reviewer, w3c-specification-writer)
• Add max-continuations: 5 to iterative code-improvement workflows
• Audit toolsets: [default] workflows and scope down where possible

Long-term (this quarter):

• Achieve 30%+ sandbox AWF coverage (currently 15%)
• Add tool-timeout: 300 to bash-heavy workflows
• Evaluate version pinning strategy for critical automated workflows

---

View Research Methodology

Analysis conducted by:

1. Examining Go source: pkg/workflow/copilot_engine*.go, copilot_mcp.go, copilot_engine_tools.go
2. Scanning all 205 workflow markdown files in .github/workflows/
3. Pattern-matching frontmatter fields via grep -rl across the workflow directory
4. Comparing against repo-memory history from 5 previous research runs (since 2026-04-17)

Data Sources: .github/workflows/*.md (205 files), Copilot engine Go source, repo-memory /tmp/gh-aw/repo-memory/default/

---

References:

• §25134300030

Generated by Copilot CLI Deep Research (Run: 25134300030)

Generated by Copilot CLI Deep Research Agent · ● 2.8M · ◷

• expires on Apr 30, 2026, 9:29 PM UTC

[github-actions[bot]]

This discussion was automatically closed because it expired on 2026-04-30T21:29:56.522Z.

Closed by Workflow