kandev/docs/specs/ci/system-design/pull-request-documentation-coverage.md

---
status: draft
last_updated: 2026-09-21
system: ci
requirements:
  - REQ-CI-PR-DOCS-001
  - REQ-CI-PR-DOCS-002
  - REQ-CI-PR-DOCS-003
  - REQ-CI-PR-DOCS-004
---

# Pull request documentation coverage system design

## Ownership and mapping

CI owns the contributor coverage policy and its trusted execution. Application systems continue to own their specifications.

| Requirement | Design sections |
| --- | --- |
| REQ-CI-PR-DOCS-001 | Classification; Artifact contract |
| REQ-CI-PR-DOCS-002 | Events and overrides |
| REQ-CI-PR-DOCS-003 | Reporting and consistency; Merge queue; Security |
| REQ-CI-PR-DOCS-004 | API read strategy and events and overrides |

## Components

Files:

- `.github/workflows/pr-docs.yml`: trusted orchestration, PR and merge-group entry points.
- `.github/scripts/pr-docs.cjs`: pure classification and reference validation, plus a bounded GitHub API adapter.
- `.github/scripts/pr-docs.test.cjs`: Node built-in test runner, with fake GitHub responses.
- `.github/scripts/pr-docs-workflow-contract_test.py`: workflow event, permission, and registration contract tests.

Register tests in `.github/workflows/lint-action-pinning.yml`, following the existing PR size workflow contract test pattern.

## Classification

Use an explicit exemption list. A changed path outside it requires coverage, including unknown roots and runtime configuration.
Evaluate both old and new paths for renames; deleting runtime code still requires coverage.

Initial exemptions:

- `docs/**` and Markdown files, including `AGENTS.md` and `CONTRIBUTING.md`.
- Go `*_test.go`; JS/TS `*.test.*` and `*.spec.*` restricted to JS/TS source extensions; `apps/web/e2e/**`.
- `apps/web/src/locales/<locale>/<namespace>.json`.
- Exact canonical registry source `plugin-registry/plugins.yaml`.
- Exact lock basenames `pnpm-lock.yaml`, `package-lock.json`, `yarn.lock`, `go.sum`, and `Cargo.lock`.
- Recognized non-Markdown harness files: `.codex/agents/*.toml`, `.codex/config.toml`, `.claude/settings.json`, and `.cursor/rules/*.mdc`.
- CI infrastructure paths `.github/workflows/**`, `.github/scripts/**`, and `.github/actions/**`. These change the delivery pipeline, not shipped product behavior, and are already governed by workflow contract tests.

Keep other `plugin-registry/**` paths subject to normal coverage. Do not exempt all JSON, YAML, assets, scripts, package manifests, Rust files, workflows, generated directories, or files containing the word `test`.
These can change shipped behavior or repository contracts. The `.github/` CI exemption is a directory-scoped rule; a workflow, script, or action outside `.github/` still requires coverage.
Add exemptions later only with concrete fixtures.
This conservative policy creates false positives for small runtime fixes and refactors. The explicit label is their escape hatch.

## Artifact contract

Select added or modified `docs/plans/<initiative>/task-<NN>-<slug>.md` files from the complete PR diff.
A pure rename with no content change does not qualify. Require at least one qualifying work order.
Read selected work orders and references at the exact PR head through the content API, as bounded text data.

Validate the repository's existing frontmatter fields: `id`, `title`, `status`, `wave`, `depends_on`, `plan`, `requirements`, `acceptance_criteria`, and `system_design`.
Require nonempty requirement, acceptance, and design lists. Resolve `plan` to the sibling `plan.md`; require a link back to the selected work order.
The plan's requirement/design references must cover the selected work order's references.
Resolve each design under `docs/specs/<system>/system-design/`. Each design
owns the subset of work-order requirements that it declares, and the union of
the referenced designs must cover every work-order requirement. Find each
owned requirement and acceptance ID in that design system's requirements
directory.
Check that AC IDs belong to referenced REQ IDs and that the design declares those REQ IDs.
Use a bounded parser for the documented frontmatter subset, not executable YAML tags or PR-supplied parsing code.

Referenced plans and specifications may already exist on the base branch. Require the work order update to record the current delivery scope and results.
Reject empty documents and references to deleted files. Allow pending work during PR development; do not require a completed implementation before a draft PR can pass structural coverage.
Semantic review determines whether changed behavior requires specification updates and whether the selected work order actually covers all changed code.
A handful of unrelated documentation changes does not establish the reference chain.

No PR-body schema is needed. The existing repository frontmatter supplies machine-readable links.
Update the PR template and contributor guide with examples and the distinction between structural coverage and semantic review.

## API read strategy

The evaluator keeps a cache for one exact pull request head or one merge-group
member. A retry after unstable pull request metadata starts a new cache. The
stable pull request path has this request budget before transient retries:

| Request class | Stable evaluation budget |
| --- | --- |
| Pull request metadata | Two reads: the initial snapshot and the consistency read. |
| Changed files | One request per 100-file page. |
| File contents | At most one read for each `(revision, path)` pair. |
| Requirement search | At most one search for each unresolved `(directory, requirement ID)` pair. |
| Commit status | One pending write and one terminal write. |

The initial pull request snapshot used to choose the pending-status revision is
also the evaluator's first snapshot. Linked work orders, plans, designs, and
requirements reuse content already loaded at the exact head.

Before code search, collect every changed Markdown requirement document in each
referenced system directory and load it at the exact head. For an existing or
renamed document, load its corresponding base path once. A referenced ID that
has one exact-head definition and the same requirement heading/ID in the
corresponding trusted base document needs no code search. The trusted base
catalog's unique-ID validation rules out an unchanged duplicate, and scanning
all changed requirement documents detects a duplicate introduced by the pull request.

Use the existing code-search and directory fallback for a new, moved, absent,
or unresolved ID. A moved ID needs fallback when it has no verified base
identity. Search results name candidates only. Read every candidate
at the exact head. Then use structural validation to establish the definition.
Missing, incomplete, or ambiguous results fail closed. Document-count, response,
and byte limits apply across the head and base reads.

## Events and overrides

Use `pull_request_target` events `opened`, `reopened`, `synchronize`, `edited`,
`labeled`, and `unlabeled` without path filters.
At the job boundary, admit label events only when the event label is exactly
`no-docs-allow`. Admit an `edited` event only when its `changes.base` field is
present, because GitHub reports base-branch retargets as edited events. Title
and description edits and draft-readiness transitions do not change any
evaluator input, so they do not start the job.
Read current PR metadata and labels, not just the event snapshot. Drafts follow the same policy.
The exact current label `no-docs-allow` returns an override success before expensive file reads.
Failure to read current metadata or labels is still an infrastructure error.

GitHub repository permissions control who applies the label. No separate author allowlist or title-based exception exists.
The override is PR-wide and persists across pushes. This is distinct from the revision-specific `ready-to-merge` contract.
Document that label actions made with `GITHUB_TOKEN` may not trigger another workflow; such automation must explicitly arrange a reevaluation.
Provide a `workflow_dispatch` PR-number input for retries, read current metadata, and validate that the PR belongs to this repository.

## Reporting and consistency

Publish one commit status context, `PR documentation coverage`, on the current PR head using `statuses: write`.
Do not rely on the native `pull_request_target` job check, whose execution identity is the base revision.
Use a distinct job name to avoid a status/check-name collision.
Set pending before evaluation; publish success, failure for missing coverage, or error for incomplete data.
The status target URL points to the run summary, which lists reasons, paths, references, and remediation.
Also fail the workflow job on policy failure or infrastructure error. Do not post PR comments.
On each retry and terminal request failure, write a bounded runner-log
diagnostic. Include the request class, HTTP status or transport category,
attempt count, and selected delay or stop reason. Do not log authorization
headers, raw response bodies, file contents, or query text.

Serialize all events by the normalized target branch with `queue: max` and
`cancel-in-progress: false`. GitHub retains up to 100 pending runs in this
shared lock; additional runs can be canceled when that bound is full. The
lock is intentionally broader than a PR-number or group-SHA lock so queued
label reevaluations cannot race with a merge-group evaluation for the same
target branch. Every surviving run reads current state.
Before publishing, reread the head, base, and label set. If changed, reevaluate with a bounded retry, then fail if state remains unstable.
Do not report stale successes for a new head. Metadata publication is eventually consistent; GitHub does not provide an atomic label-read/status-write transaction.

Paginate file lists and compare the count to current `changed_files`. Reject results at GitHub's 3,000-file cap and mismatched counts.
Reject truncated responses, invalid encodings, oversized documents, unsupported frontmatter, symlinks, submodules, and ambiguous IDs.
Bound artifact reads to 100 documents, 256 KiB each, and 4 MiB total. Report limits explicitly; never silently discard documents.

The API adapter makes at most three attempts for transient transport failures,
HTTP 408 or 429, and retryable 5xx responses. It also retries HTTP 403 responses
that GitHub identifies as rate limiting and successful merge-queue GraphQL
responses with an error of type `RATE_LIMITED`. Other 4xx responses and other
GraphQL errors fail immediately. A usable `Retry-After` value takes
precedence. A primary limit with no remaining quota uses `X-RateLimit-Reset`.
A secondary limit without usable guidance waits 60 seconds before the second
attempt and 120 seconds before the third. Transport, 408, and 5xx failures use
short exponential backoff. The 180-second sleep budget is shared across all
requests in one client evaluation. A server wait beyond the remaining budget
fails instead of exceeding the job timeout.

Each attempt has its own request timeout. A status retry uses the same revision,
context, state, and target URL. An uncertain response can create a duplicate
status record. It cannot change the intended result. Request retries
do not replace the separate bounded reevaluation used when pull request metadata
changes.

## Merge queue

Handle `merge_group: checks_requested` and publish the same status on `merge_group.head_sha`.
Load only trusted base code. Never evaluate coverage against the combined diff, where one PR could supply another's artifacts.

Resolve members using the paginated GraphQL merge queue entries, including each entry's `baseCommit`, `headCommit`, and `pullRequest`.
Find the entry for the event head and trace the entry commit boundaries back to the event base.
Require a complete, unambiguous chain; unknown membership produces an error, not success.
Evaluate each member's own current PR diff, artifact chain, and labels. Confirm queue identities and member heads again before publication.
Use the target-branch concurrency key shared with queued label reevaluation and
never cancel active group evaluation. The group status itself remains on the
synthetic group SHA.

Integration must validate the entry-boundary mapping against real GitHub merge-group payloads before requiring this status.
If GitHub cannot supply the expected chain for a supported queue mode, revise the mapping and fixtures before rollout.
Label removal while queued must also reevaluate every active group prefix
containing that PR, through the same group evaluator and serialization key.
Do not dequeue, requeue, or automatically merge PRs.

## Security

Permissions are `contents: read`, `pull-requests: read`, and `statuses: write` only.
Use SHA-pinned actions, GitHub-hosted runners, a trusted base checkout, and `persist-credentials: false`.
For dispatch, restrict the workflow definition to the default branch. For queue runs, pin consumed scripts to the event base SHA.
Do not install PR dependencies, execute PR files, or interpolate PR text into shell or JavaScript source.
Normalize artifact paths inside their allowed roots, bind content requests to exact repository/head identities, and escape summary text.

## Rollout and persistence

GitHub stores labels, statuses, and run summaries. No application storage or feature flag is needed.
First merge the workflow and contributor instructions. Exercise pass, failure, override, removal, fork, and merge-group cases.
Then add the status context to the `main` ruleset as a separate administrator action, preserving all existing checks and bypass rules.
Until required, it is a failing PR check but does not itself prohibit merge. Do not report ruleset enforcement as deployed before verification.
Existing open PRs need a supported event or dispatch to receive their first result.

## Related decisions and references

- [Documentation coverage policy](../../../decisions/2026-09-10-pr-documentation-coverage.md)
- [Existing merge queue contract](../../../ci-merge-queue.md)
- [GitHub workflow events](https://docs.github.com/en/actions/reference/workflows-and-actions/events-that-trigger-workflows)
- [GitHub required status checks](https://docs.github.com/en/pull-requests/how-tos/merge-and-close-pull-requests/troubleshooting-required-status-checks)
- [GitHub merge queue API fields](https://docs.github.com/en/graphql/reference/pulls)
- [GitHub REST API rate limits](https://docs.github.com/en/rest/using-the-rest-api/rate-limits-for-the-rest-api)
- [GitHub REST API best practices](https://docs.github.com/rest/guides/best-practices-for-integrators)