Enhance triage skills with updated scoring, confidence, and P-scale labels#6294
Enhance triage skills with updated scoring, confidence, and P-scale labels#6294lauren-ciha wants to merge 11 commits intomainfrom
Conversation
guimafelipe
left a comment
guimafelipe
left a comment
There was a problem hiding this comment.
As this is a script that we use internally, I don't think any of my comments should be blocking. But maybe they will help making this overall process more maintainable and consistent.
Feel free to take them as suggestions.
|
|
||
| Normalized Score = (Total Score / 100) × 100 | ||
| Total Score = Reactions + Age + Comments + Severity | ||
| = 30 + 30 + 30 + 10 = 100 max |
There was a problem hiding this comment.
I would like to review that later. I feel like severity should have a way bigger impact on the score. If we have a security issue, we want to act on it ASAP regardless of reactions, comments and age.
There was a problem hiding this comment.
I'm going to loop in @ssparach on this one. I'm open to updating the criteria.
There was a problem hiding this comment.
Severity handling needs to be adjusted. Once severity is appropriately tracking actual security and data loss issues, its weight can be increased to reflect their higher priority
| $defaultKeywords = @{ | ||
| 'area-Notifications' = @('notification', 'toast', 'badge', 'push', 'appnotification', 'pushnotification', 'wns') | ||
| 'area-Packaging' = @('msix', 'package', 'deploy', 'install', 'appx', 'deployment') | ||
| 'area-Windowing' = @('window', 'appwindow', 'titlebar', 'backdrop', 'presenter') | ||
| 'area-Widgets' = @('widget', 'dashboard') | ||
| 'area-AppLifecycle' = @('lifecycle', 'activation', 'restart', 'single instance', 'appinstance') | ||
| 'area-PowerManagement' = @('power', 'battery', 'suspend', 'resume', 'powermanager') | ||
| 'area-MRTCore' = @('resource', 'mrt', 'localization', 'pri', 'resourcemanager') | ||
| 'area-DWriteCore' = @('font', 'dwrite', 'text', 'typography') | ||
| 'area-AccessControl' = @('access', 'security', 'token', 'permission') | ||
| 'area-Environment' = @('environment', 'variable', 'env') | ||
| } |
There was a problem hiding this comment.
Should this go in the json file too?
There was a problem hiding this comment.
Great question! I think instead of having the defaults here, we should rely on the output of .github\skills\triage-meeting-prep\scripts\Get-RepositoryLabels.ps1. I'll update those shortly.
| | None of above | 0 | | ||
| | Severity Tier | Labels | Score | | ||
| |---------------|--------|-------| | ||
| | Critical | `regression`, `crash`, `hang`, `data-loss`, `security`, `P0` | 10 | |
There was a problem hiding this comment.
Do we plan to add labels for crash, hang, and data loss and some others? These labels do not exist today.
Also, regarding the security label, the skill is currently referencing area-security, which is intended to track bugs related to OAuth2Manager APIs. Severity falls outside that scope, so a different area label should be used here. Happy to align on which label makes the most sense. Thanks!
| # area-Notifications covers all notification types (toast, badge, push, app notifications) | ||
| $defaultKeywords = @{ | ||
| 'area-Notifications' = @('notification', 'toast', 'badge', 'push', 'appnotification', 'pushnotification', 'wns') | ||
| 'area-Packaging' = @('msix', 'package', 'deploy', 'install', 'appx', 'deployment') |
There was a problem hiding this comment.
Keywords such as msix and appxdeployment would be more appropriate for the area-PackageManagement label.
| 'area-PowerManagement' = @('power', 'battery', 'suspend', 'resume', 'powermanager') | ||
| 'area-MRTCore' = @('resource', 'mrt', 'localization', 'pri', 'resourcemanager') | ||
| 'area-DWriteCore' = @('font', 'dwrite', 'text', 'typography') | ||
| 'area-AccessControl' = @('access', 'security', 'token', 'permission') |
There was a problem hiding this comment.
It looks like the area-AccessControl and area-Environment labels do not currently exist. We may want to align on the appropriate existing area labels instead for keyword mapping.
…abels
Scoring System:
- New weights: reactions=30%, age=30%, comments=30%, severity=10%
- P-scale severity labels: P0=critical, P1=high, P2=medium, P3=low
- Confidence scoring with grep-friendly [confidence:XX] format (0-100)
Label Consolidation:
- Merged Hot + Popular into Popular (>=5 reactions threshold)
Contact Schema:
- Simplified from {primary, secondary} to single {contact} field
- Removed legacy schema backward compatibility
Area Suggestions:
- Get-IssueDetails.ps1 dynamically fetches area labels via Get-RepositoryLabels.ps1
- area-Notifications covers all notification types (toast, badge, push, wns)
- Fixed area-PowerManagement naming
Documentation:
- Added PowerShell examples alongside Bash for confidence filtering
- Updated all SKILL.md files with new configuration details
lauren-ciha
force-pushed
the
user/laurenciha/weekly-sync-scoring
branch
2 times, most recently
from
1fcc9b3 to
c0833b4
Compare
…onfig.json for severity labels
… fails fast on errors
…script handle human-readable formatting
…s the source of truth
3 participants
This PR builds upon the issue-triage and feature-area-report skills.
The key changes are:
Validate-FeatureAreaReport.ps1that checks the report against the live GitHub label data usingghcli