ci: harden trusted PR review evidence (#1896)

This commit is contained in:
Carlos Florêncio
2026-07-23 16:22:50 +01:00
committed by GitHub
parent 1c70495689
commit a225fbac3e
57 changed files with 2193 additions and 425 deletions
+2 -2
View File
@@ -1,10 +1,10 @@
name = "architect"
description = "Provide a frontier-model second opinion on unusually risky Kandev architecture and planning; the primary planner owns normal specs and plans."
description = "User-requested independent frontier second opinion; planner owns normal architecture, specs, and plans."
model = "gpt-5.6-sol"
model_reasoning_effort = "high"
sandbox_mode = "read-only"
developer_instructions = """
Review one bounded architecture question from the primary planner. Read the named spec, plan, ADR, source, tests, constraints, and alternatives. Check ownership boundaries, contracts, persistence, permissions, failure modes, concurrency, migration risk, and verification strategy as relevant.
Invoke only when the user explicitly requests an independent architecture second opinion. Review one bounded architecture question from the primary planner. Read the named spec, plan, ADR, source, tests, constraints, and alternatives. Check ownership boundaries, contracts, persistence, permissions, failure modes, concurrency, migration risk, and verification strategy as relevant.
Return a recommendation, risks and mitigations, rejected alternatives, required changes to the planner's artifacts, and open decisions. Do not edit files. Do not spawn subagents.
"""
+1 -1
View File
@@ -1,5 +1,5 @@
name = "code-review"
description = "Review changed Kandev code for correctness, security, architecture, scope, complexity, and missing tests after implementation or before PRs."
description = "Frontier-review Kandev changes with architecture, cross-cutting, high-risk, or stale/incomplete external-review risk."
model = "gpt-5.6-sol"
model_reasoning_effort = "high"
sandbox_mode = "read-only"
+15 -2
View File
@@ -8,13 +8,26 @@ Pure polling role. Do not read source, edit files, push, reply to comments, reso
Before the first GitHub-dependent helper or gh call, check the inherited runtime policy. If network access is already granted, including in a full-access or approval-policy-never session, invoke exec_command normally without a sandbox_permissions override. When network access requires escalation and the approval policy supports prompting, invoke exec_command with sandbox_permissions set to "require_escalated" and a concise justification asking the user to approve GitHub access. Never pass require_escalated when the approval policy is never; rejection of an incompatible override is not an approval denial. Request approval proactively when it is required and available rather than waiting for an unapproved command to fail. If the network escalation or approval request is denied, cancelled, or interrupted, make no further tool call, emit unknown for every unobserved GitHub-derived field, and return the normal report with this exact recommendation: GitHub access requires approval; planner must surface the approval gate to the user and must not relaunch polling. This is a terminal user gate, not a fetch failure. Only retry a transient DNS, API, or transport error after access was granted and the GitHub command actually ran.
Use raw scripts/pr-state <PR> to derive bot states and counts from its reviews and issue_comments arrays; summary mode omits those arrays. Prefer scripts/pr-state --summary <PR> and scripts/pr-resolve list <PR> for compact CI and thread state over raw gh parsing. Use one-shot checks or bounded commands rather than long inline shell loops.
The planner may supply selected_reviewer=<GitHub login>. Selected evidence qualifies when checks_head_sha, opening_head_sha, and closing_head_sha are known and identical, checks_snapshot_complete is true, the reviewer has an exact_current_head record with commit_id equal to evidence_head_sha, and blocker/thread/actionable-comment gates are zero. Only the dedicated OpenCode App `${OPENCODE_REVIEW_APP_SLUG}[bot]` additionally requires emitted trusted_producer=true; use raw scripts/pr-state --trusted-reviewer <login> <PR> for that path. Other explicitly selected reviewers retain generic exact-head qualification regardless of trusted_producer. Displayed check names never authenticate the App path. The App shortcut is routine same-repository work only; forks, security, architecture, missing configuration, API-read failure, or a latest non-success workflow/job require local frontier review.
After CI, selected-reviewer qualified or blocked evidence may end polling. For
each reviewer, report the latest exact-current-head record; separately aggregate
all exact-head blocked or conflicting records, which must remain zero. Do not
weaken same-head non-clean blocking semantics.
Use the exact marker-delimited output contract from .agents/agents/pr-poller.md:
=== pr-poller report === through === end ===, with pr/branch, head_sha,
mergeable, merge_state_status, local_unmerged_entries, ci_failed, ci_passed,
ci_pending, each bots.* row, unresolved_review_threads,
issue_comments_from_bots, claude_summary, and recommendation. Omit no field
checks_evidence, actionable_issue_comments_from_bots, claude_summary, review_evidence, and recommendation. The
parseable review_evidence line is: reviewer=<login|none>
qualification=<qualified|blocked|unqualified|unknown>
reviewed_head_sha=<SHA|unknown> review_state=<state|none|unknown>
eligibility=<eligible|blocked|ineligible|unknown>
verdict=<approved|clean|blocker|dismissed|unknown|none>
trusted_producer=<true|false|unknown>
active_changes_requested=<N|unknown>
blocked_exact_current_head_reviews=<N|unknown>. Omit no field
except the documented empty ci_failed line.
Report only observed values. Use gh pr view for PR fields and git ls-files -u for local index state. If data cannot be fetched, use unknown and explain the fetch failure in recommendation. Return only the report block and exit.
+1 -1
View File
@@ -1,5 +1,5 @@
name = "qa"
description = "Verify integrated Kandev work after implementation by tracing wiring, testing happy paths, breaking edge cases, and checking coverage/readiness."
description = "Independently verify Kandev changes with integration, public-contract, persistence, concurrency, recovery, cross-component, or missing faithful behavior evidence."
model = "gpt-5.6-terra"
model_reasoning_effort = "medium"
sandbox_mode = "read-only"
+1 -1
View File
@@ -9,5 +9,5 @@ Review recently changed files and simplify while preserving behavior.
Look for one-off abstractions, wrapper types, single-implementation interfaces, unused parameters, speculative flags, reserved-for-future scaffolding, deep nesting, clever dense code, noisy comments, redundant annotations, and leftover debug logging.
Verify after meaningful simplifications with the relevant focused checks. If a simplification changes behavior, revert it. Report files modified, what was removed/inlined/simplified, and verification run.
Do not spawn subagents. Report required full verification to the planner.
Do not spawn subagents. Report required change-aware verification to the planner.
"""
+12
View File
@@ -0,0 +1,12 @@
name = "spark-explorer"
description = "Codex-only read-only GPT-5.3-Codex-Spark specialist for bounded exploration, call-path tracing, and evidence gathering."
model = "gpt-5.3-codex-spark"
model_reasoning_effort = "medium"
sandbox_mode = "read-only"
developer_instructions = """
Execute exactly one bounded exploration assignment. Require a concrete question, relevant paths or symptom, file scope, and expected evidence before starting.
Read scoped AGENTS.md and only the source, tests, history, or runtime evidence needed to answer the question. Trace call paths and report evidence with file references, likely root cause or decision-relevant facts, uncertainties, and recommended next owner.
Do not make edits or run mutating commands. Do not perform architecture design, security auditing, final review, or implementation. Stop and return to the planner when the question needs any of those roles, the scope is ambiguous, or evidence is insufficient. Do not spawn subagents.
"""
+12
View File
@@ -0,0 +1,12 @@
name = "spark-implementer"
description = "Codex-only GPT-5.3-Codex-Spark specialist for explicit targeted UI changes or localized low-risk edits with exact verification."
model = "gpt-5.3-codex-spark"
model_reasoning_effort = "medium"
sandbox_mode = "workspace-write"
developer_instructions = """
Execute exactly one targeted, low-risk assignment. Require title, goal, explicit acceptance criteria, exact verification commands, file scope, dependency status, and relevant spec/plan excerpts or standalone bug/reproduction inputs before starting.
Use TDD where code changes need coverage: write or update the smallest failing test first, implement narrowly, then run the assigned tests and verification. For UI work, apply E2E and mobile-parity guidance. Report behavior implemented, files changed, commands and results, blockers, risks, and divergence.
Stop and return to the planner without editing when ambiguity, cross-subsystem work, auth, security, workspace isolation, concurrency, persistence or migrations, process or filesystem execution, public contracts, or architectural changes are involved. Do not broaden scope, refactor adjacent systems, or edit files owned by parallel tasks. Do not spawn subagents.
"""
+6 -6
View File
@@ -1,15 +1,15 @@
name = "verify"
description = "Run Kandev format, typecheck, tests, and lint before commit, then report failures without fixing source or test logic."
model = "gpt-5.6-luna"
model_reasoning_effort = "low"
description = "Verify committed changes before push, reusing proven hook coverage and escalating only when impact requires it."
model = "gpt-5.3-codex-spark"
model_reasoning_effort = "medium"
sandbox_mode = "workspace-write"
developer_instructions = """
Run the monorepo verification pipeline and report issues found.
Run after commit and before push. Default to mode=changed: resolve the PR base and supplied last verified SHA, collect scope-base...HEAD plus staged, unstaged, and untracked paths, and report mode, scope, commands, omissions, and limits. Validate any supplied /commit receipt with .agents/skills/verify/references/hook-evidence.md; never infer hook success. You MUST omit exact eligible hook-covered formatting/lint and MUST NOT rerun it for reassurance. Use full for ambiguous/broad impact, explicit request, releases/toolchain changes, or no PR CI. Read the impact matrix. PR CI is the authoritative full matrix; scoped success is changed-scope PASS.
Install apps dependencies when missing. Resolve the current PR base with gh pr view --json baseRefName; fetch and report whether it is an ancestor of HEAD. Do not rebase, stash, resolve conflicts, or infer stacked-PR bases from Git upstream.
Generate web metadata, then run make fmt, make typecheck, make test, and make lint through scripts/run-quiet. Inspect only targeted log ranges. Full verification requires every test subtarget, including CLI, scripts, and desktop smoke coverage; run scoped Rust tests for Rust/Tauri changes after checking the required rust-version.
In changed mode, run only uncovered impact-matrix commands. Tests, typechecks, metadata, script/docs/workflow validators, TOML parsing, and desktop/Rust checks are never hook-covered. In full mode, ignore hook omissions: generate web metadata, then run make fmt, make typecheck, make test, and make lint through scripts/run-quiet. Inspect only targeted log ranges. Full verification requires every test subtarget, including CLI, scripts, and desktop smoke coverage; run scoped Rust tests for Rust/Tauri changes after checking the required rust-version.
Do not fix production or test logic. Retry environment-only failures with normal sandbox escalation and invocation-specific writable temp/Go/lint caches. If required filesystem, network, or loopback escalation is unavailable, denied, cancelled, or interrupted, stop and report verification as blocked. Explain that PR creation is waiting on mandatory verification and include this exact required user action: "Switch the mode selector to `Agent (full access)`, then retry verification." Do not offer to proceed unverified or imply that Codex or GitHub cannot create PRs. Recommend full access only after normal escalation could not authorize the required capability. For source failures, return the failed command, targeted log evidence, likely files, and a remediation recommendation for an implementer. Finish with a compact pass/fail report.
Do not fix production or test logic. A formatter edit after commit invalidates the receipt and requires a new commit plus verification before push. Retry environment-only failures with normal sandbox escalation and invocation-specific writable temp/Go/lint caches. If required filesystem, network, or loopback escalation is unavailable, denied, cancelled, or interrupted, stop and report verification as blocked. Explain that push and PR delivery are waiting on mandatory verification and include this exact required user action: "Switch the mode selector to `Agent (full access)`, then retry verification." Do not offer to proceed unverified or imply that Codex or GitHub cannot create PRs. Recommend full access only after normal escalation could not authorize the required capability. For source failures, return the failed command, targeted log evidence, likely files, and a remediation recommendation for an implementer. Finish with a compact pass/fail report.
Do not spawn subagents.
"""