docs(merge-gate): require gate query and merge as separate invocations#67
Conversation
A PR was merged while 3 review threads were still unresolved because the
merge-gate query and gh pr merge were chained in one compound shell
command (gate-query && gh pr merge). Shell chaining decides on exit
codes, not on the gate's content: gh pr view exits 0 regardless of how
many unresolved threads it reports, so the merge executed before the
gate output ("unresolved: 3") was ever read. The threads contained a
high-severity security finding that required a follow-up PR.
Add an explicit rule to the Merge Gate section: run the gate query, read
its output, and only then issue gh pr merge as a new command — never
chain them. Also note that mergeStateStatus CLEAN does not imply zero
unresolved threads (GitHub only couples those when the "require
conversation resolution" branch-protection rule is enabled), plus a
wrong/right example pair and an eval asserting the two-invocation
behavior.
Signed-off-by: Sebastian Mendel <github@sebastianmendel.de>
Dependency Review✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.Scanned FilesNone |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a new evaluation test case (merge_gate_separate_invocation) and updates the pull request workflow documentation to emphasize that running a merge gate check and executing the merge must be done in separate invocations rather than chained together (e.g., via &&). The review feedback suggests expanding the assertion regex patterns in the new evaluation test case to be more robust and prevent false negatives by matching more variations of refusal words and terms like "conversation" instead of just "thread".
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
Match additional refusal verbs (cannot, should not, must not) and common phrasing variants (exits with 0, unresolved conversations) to reduce false negatives in the merge_gate_separate_invocation eval. Signed-off-by: Sebastian Mendel <github@sebastianmendel.de>
|
Why
Incident (2026-06-12): a PR was merged while 3 review threads were still unresolved. The merge-gate query and
gh pr mergehad been chained in one compound shell command (gate-query && gh pr mergestyle). The gate's output ("unresolved: 3") only became visible after the merge had already executed. The unresolved threads contained a high-severity security finding (worker running as root) that then required a follow-up PR.The root cause is structural, not a one-off mistake: shell chaining decides on exit codes, not on the gate's content.
gh pr viewexits 0 whether it reports zero unresolved threads or three, so anyquery && merge(or query-then-merge inside one heredoc/compound command) merges unconditionally. ACLEANmergeStateStatusdoes not close this gap either — GitHub only couples merge state to thread resolution when the "require conversation resolution" branch-protection rule is enabled, which most repos don't turn on.What
skills/git-workflow/references/pull-request-workflow.md(Merge-Gate Command section): new rule — the gate and the merge are two separate invocations; run the gate query, read its output, only then issuegh pr mergeas a new command. Includes a wrong/right example pair in the file's existing style.skills/git-workflow/evals/evals.json: new evalmerge_gate_separate_invocation— given a PR with green CI,CLEANmerge state, and possibly-open threads, the assistant must not merge in the same command as the gate query.Validation
pre-commit run --all-files: all hooks pass on the changed files; the only failures are two pre-existing shellcheck findings in untouched files (.envrcSC2148,verify-git-workflow.shSC2001), present onmainbefore this change.