Carlos Florêncio
9.9 KiB
status, system, requirements
| status | system | requirements | |||
|---|---|---|---|---|---|
| draft | ci |
|
Pull request documentation coverage system design
Ownership and mapping
CI owns the contributor coverage policy and its trusted execution. Application systems continue to own their specifications.
| Requirement | Design sections |
|---|---|
| REQ-CI-PR-DOCS-001 | Classification; Artifact contract |
| REQ-CI-PR-DOCS-002 | Events and overrides |
| REQ-CI-PR-DOCS-003 | Reporting and consistency; Merge queue; Security |
Components
Proposed files:
.github/workflows/pr-docs.yml: trusted orchestration, PR and merge-group entry points..github/scripts/pr-docs.cjs: pure classification and reference validation, plus a bounded GitHub API adapter..github/scripts/pr-docs.test.cjs: Node built-in test runner, with fake GitHub responses..github/scripts/pr-docs-workflow-contract_test.py: workflow event, permission, and registration contract tests.
Register tests in .github/workflows/lint-action-pinning.yml, following the existing PR size workflow contract test pattern.
Classification
Use an explicit exemption list. A changed path outside it requires coverage, including unknown roots and runtime configuration. Evaluate both old and new paths for renames; deleting runtime code still requires coverage.
Initial exemptions:
docs/**and Markdown files, includingAGENTS.mdandCONTRIBUTING.md.- Go
*_test.go; JS/TS*.test.*and*.spec.*restricted to JS/TS source extensions;apps/web/e2e/**. apps/web/src/locales/<locale>/<namespace>.json.- Exact lock basenames
pnpm-lock.yaml,package-lock.json,yarn.lock,go.sum, andCargo.lock. - Recognized non-Markdown harness files:
.codex/agents/*.toml,.codex/config.toml,.claude/settings.json, and.cursor/rules/*.mdc.
Do not exempt all JSON, YAML, assets, scripts, package manifests, Rust files, workflows, generated directories, or files containing the word test.
These can change shipped behavior or repository contracts. Add exemptions later only with concrete fixtures.
This conservative policy creates false positives for small runtime fixes and refactors. The explicit label is their escape hatch.
Artifact contract
Select added or modified docs/plans/<initiative>/task-<NN>-<slug>.md files from the complete PR diff.
A pure rename with no content change does not qualify. Require at least one qualifying work order.
Read selected work orders and references at the exact PR head through the content API, as bounded text data.
Validate the repository's existing frontmatter fields: id, title, status, wave, depends_on, plan, requirements, acceptance_criteria, and system_design.
Require nonempty requirement, acceptance, and design lists. Resolve plan to the sibling plan.md; require a link back to the selected work order.
The plan's requirement/design references must cover the selected work order's references.
Resolve each design under docs/specs/<system>/system-design/. Each design
owns the subset of work-order requirements that it declares, and the union of
the referenced designs must cover every work-order requirement. Find each
owned requirement and acceptance ID in that design system's requirements
directory.
Check that AC IDs belong to referenced REQ IDs and that the design declares those REQ IDs.
Use a bounded parser for the documented frontmatter subset, not executable YAML tags or PR-supplied parsing code.
Referenced plans and specifications may already exist on the base branch. Require the work order update to record the current delivery scope and results. Reject empty documents and references to deleted files. Allow pending work during PR development; do not require a completed implementation before a draft PR can pass structural coverage. Semantic review determines whether changed behavior requires specification updates and whether the selected work order actually covers all changed code. A handful of unrelated documentation changes does not establish the reference chain.
No PR-body schema is needed. The existing repository frontmatter supplies machine-readable links. Update the PR template and contributor guide with examples and the distinction between structural coverage and semantic review.
Events and overrides
Use pull_request_target events opened, reopened, synchronize, edited, labeled, unlabeled, and ready_for_review without path filters.
Read current PR metadata and labels, not just the event snapshot. Drafts follow the same policy.
The exact current label no-docs-allow returns an override success before expensive file reads.
Failure to read current metadata or labels is still an infrastructure error.
GitHub repository permissions control who applies the label. No separate author allowlist or title-based exception exists.
The override is PR-wide and persists across pushes. This is distinct from the revision-specific ready-to-merge contract.
Document that label actions made with GITHUB_TOKEN may not trigger another workflow; such automation must explicitly arrange a reevaluation.
Provide a workflow_dispatch PR-number input for retries, read current metadata, and validate that the PR belongs to this repository.
Reporting and consistency
Publish one commit status context, PR documentation coverage, on the current PR head using statuses: write.
Do not rely on the native pull_request_target job check, whose execution identity is the base revision.
Use a distinct job name to avoid a status/check-name collision.
Set pending before evaluation; publish success, failure for missing coverage, or error for incomplete data.
The status target URL points to the run summary, which lists reasons, paths, references, and remediation.
Also fail the workflow job on policy failure or infrastructure error. Do not post PR comments.
Serialize all events by the normalized target branch with queue: max and
cancel-in-progress: false. GitHub retains up to 100 pending runs in this
shared lock; additional runs can be canceled when that bound is full. The
lock is intentionally broader than a PR-number or group-SHA lock so queued
label reevaluations cannot race with a merge-group evaluation for the same
target branch. Every surviving run reads current state.
Before publishing, reread the head, base, and label set. If changed, reevaluate with a bounded retry, then fail if state remains unstable.
Do not report stale successes for a new head. Metadata publication is eventually consistent; GitHub does not provide an atomic label-read/status-write transaction.
Paginate file lists and compare the count to current changed_files. Reject results at GitHub's 3,000-file cap and mismatched counts.
Reject truncated responses, invalid encodings, oversized documents, unsupported frontmatter, symlinks, submodules, and ambiguous IDs.
Bound artifact reads to 100 documents, 256 KiB each, and 4 MiB total. Report limits explicitly; never silently discard documents.
Merge queue
Handle merge_group: checks_requested and publish the same status on merge_group.head_sha.
Load only trusted base code. Never evaluate coverage against the combined diff, where one PR could supply another's artifacts.
Resolve members using the paginated GraphQL merge queue entries, including each entry's baseCommit, headCommit, and pullRequest.
Find the entry for the event head and trace the entry commit boundaries back to the event base.
Require a complete, unambiguous chain; unknown membership produces an error, not success.
Evaluate each member's own current PR diff, artifact chain, and labels. Confirm queue identities and member heads again before publication.
Use the target-branch concurrency key shared with queued label reevaluation and
never cancel active group evaluation. The group status itself remains on the
synthetic group SHA.
Integration must validate the entry-boundary mapping against real GitHub merge-group payloads before requiring this status. If GitHub cannot supply the expected chain for a supported queue mode, revise the mapping and fixtures before rollout. Label removal while queued must also reevaluate every active group prefix containing that PR, through the same group evaluator and serialization key. Do not dequeue, requeue, or automatically merge PRs.
Security
Permissions are contents: read, pull-requests: read, and statuses: write only.
Use SHA-pinned actions, GitHub-hosted runners, a trusted base checkout, and persist-credentials: false.
For dispatch, restrict the workflow definition to the default branch. For queue runs, pin consumed scripts to the event base SHA.
Do not install PR dependencies, execute PR files, or interpolate PR text into shell or JavaScript source.
Normalize artifact paths inside their allowed roots, bind content requests to exact repository/head identities, and escape summary text.
Rollout and persistence
GitHub stores labels, statuses, and run summaries. No application storage or feature flag is needed.
First merge the workflow and contributor instructions. Exercise pass, failure, override, removal, fork, and merge-group cases.
Then add the status context to the main ruleset as a separate administrator action, preserving all existing checks and bypass rules.
Until required, it is a failing PR check but does not itself prohibit merge. Do not report ruleset enforcement as deployed before verification.
Existing open PRs need a supported event or dispatch to receive their first result.