docs: enforce test requirements and improve agent skill resilience (#543)

This commit is contained in:
Carlos Florêncio
2026-04-09 16:27:19 +01:00
committed by GitHub
parent d8ce686f4e
commit ebefac814a
7 changed files with 32 additions and 11 deletions
+7 -6
View File
@@ -71,11 +71,12 @@ Check every changed file for the following layers. Skip layers that don't apply
- `as any` or `as unknown as X` casts used to dodge type errors instead of fixing types
- Defensive checks abnormal for the area of the codebase — compare with surrounding code patterns
**Testing:**
- Backend (Go): new or changed functions/methods should have corresponding tests
- Frontend (JS/TS libs only): new utility functions, hooks, API clients, and store slices should have tests
**Testing (blocker if missing):**
- Backend (Go): new or changed functions/methods must have corresponding `*_test.go` tests
- Frontend (JS/TS libs only): new utility functions, hooks, API clients, and store slices must have `*.test.ts` tests
- We do NOT test React components — skip those
- Flag untested logic but don't block on it; suggest what tests to add and recommend `/tdd`
- Exceptions: config files, generated code, React component markup
- Missing tests for new or changed logic is a **blocker** — suggest what tests to add and recommend `/tdd`
### 3. Fix or report
@@ -91,7 +92,7 @@ Use this format:
### Findings
#### Blocker (must fix before merge)
*Security holes, data loss risk, broken logic, crashes*
*Security holes, data loss risk, broken logic, crashes, missing tests for new/changed logic*
1. **[Title]** — `file.go:42`
- Issue: what's wrong
@@ -99,7 +100,7 @@ Use this format:
- Fix: concrete suggestion or code snippet
#### Suggestion (recommended, doesn't block)
*Performance problems, poor error handling, architectural concerns, missing tests*
*Performance problems, poor error handling, architectural concerns*
### Summary
+2 -2
View File
@@ -82,7 +82,7 @@ Before writing an E2E test, validate the feature works interactively using `play
### Start the dev environment
Multiple agents may run in parallel, so use random ports to avoid collisions. Fixture ports use 18080-18089 (backend) and 13000-13009 (frontend) — stay outside those ranges.
Multiple agents may run in parallel, so use random ports to avoid collisions. Fixture ports auto-offset from 18080 (backend) and 13000 (frontend) using `E2E_PORT_OFFSET` (derived from `PID % 30` by default) — stay outside those ranges. Parallel E2E test runs are safe by default.
```bash
OFFSET=$((RANDOM % 100))
@@ -217,7 +217,7 @@ When a test fails:
- **"Backend did not become healthy"** — run `make build-backend build-web`, check with `E2E_DEBUG=1`
- **"Cannot find module"** — run `cd apps && pnpm install`
- **Port conflicts** — backends use 18080+, frontends use 13000+ (per worker)
- **Port conflicts** — backends use 18080+ and frontends use 13000+ (per worker), auto-offset by `E2E_PORT_OFFSET` (derived from PID). Set `E2E_PORT_OFFSET=0` for deterministic ports
- **Flaky timeouts** — **never increase locator timeouts to fix flaky tests.** If a locator times out, the root cause is almost always something else: a setup failure, missing navigation, race condition, or the element genuinely not rendering. Investigate why the element never appears instead of giving it more time. Note: infrastructure health timeouts (30s in `fixtures/backend.ts`) and overall test timeouts (60s in `playwright.config.ts`) are separate and should not be modified either.
- Screenshots on failure, video on first retry (CI)
+2
View File
@@ -7,6 +7,8 @@ description: Wait for CI checks and automated reviews (CodeRabbit, Greptile, Cla
Wait for CI and code review to complete on a pull request, fix any failures or valid comments, then push.
> **GitHub tool selection:** This skill uses `gh` CLI commands by default. If `gh` is unavailable or fails, use any available GitHub tools in the environment (e.g. MCP GitHub tools) for PR checks, comments, replies, and reviews. Some operations (reactions, resolving threads, fetching CI logs) may not be available in all environments — skip gracefully.
## Available skills and subagents
- **`verify` subagent** — Run the full verification pipeline (format, typecheck, test, lint) before pushing fixes.
+2
View File
@@ -5,6 +5,8 @@ description: Commit, push, and create a PR. Default is ready-for-review with aut
# PR
> **GitHub tool selection:** This skill uses `gh` CLI commands by default. If `gh` is unavailable or fails, use any available GitHub tools in the environment (e.g. MCP GitHub tools) to create, edit, and view the PR. The goal is the same — the tool may differ.
## Available skills
- **`/commit`** — Stage and commit changes using Conventional Commits. Runs `/verify` internally.
+2
View File
@@ -19,6 +19,8 @@ description: Commit and push to the current branch. Use --fixup to also wait for
- `--fixup` — after pushing, run `/pr-fixup` to wait for CI and CodeRabbit review, fix issues, and push again.
> **Note:** This skill only uses `git push`. GitHub CLI dependency is indirect via `/pr-fixup`.
## Your task
Commit any pending changes and push to the remote branch.
+7
View File
@@ -287,6 +287,10 @@ Static analysis runs in CI and pre-commit. New code **must** stay within these l
**When you hit a limit:** extract a helper function, custom hook, or sub-component. Prefer composition over growing a single function.
### Testing
Every code change must include tests for new or changed logic. Backend: `*_test.go` files alongside the source. Frontend: `*.test.ts` files for utility functions, hooks, API clients, and store slices. Exceptions: config files, generated code, React component markup. Use `/tdd` for test-driven development.
### Knowledge
- **Decisions:** Architecture decisions are recorded in `docs/decisions/`. Read `docs/decisions/INDEX.md` for an overview. When making significant architectural choices, create a new ADR via `/record decision`.
- **Plans:** Implementation plans are stored in `docs/plans/`. Save completed feature designs via `/record plan`.
@@ -313,6 +317,9 @@ Static analysis runs in CI and pre-commit. New code **must** stay within these l
### Plan Implementation
- After implementing a plan, run `make fmt` first to format code, then run `make typecheck test lint` to verify the changes. Formatting must come first because formatters may split lines, which can trigger complexity linter warnings.
### GitHub Operations
Skills use `gh` CLI by default. If a `gh` command fails (not installed, not authenticated, etc.), use whatever GitHub tools are available in the environment (MCP GitHub tools, API tools, etc.) to accomplish the same operation. The goal is the same — the tool may differ.
---
## Maintaining This File
+10 -3
View File
@@ -10,8 +10,15 @@ const KANDEV_BIN = path.join(BACKEND_DIR, "bin", "kandev");
const STANDALONE_SERVER = path.join(WEB_DIR, ".next/standalone/web/server.js");
const STANDALONE_STATIC_DIR = path.join(WEB_DIR, ".next/standalone/web/.next/static");
const SOURCE_STATIC_DIR = path.join(WEB_DIR, ".next/static");
const BACKEND_BASE_PORT = 18080;
const FRONTEND_BASE_PORT = 13000;
// Auto-derive from PID if not explicitly set — prevents port clashes between concurrent test runs
// Modulo 30 keeps agentctl ports under 65535 (30001 + 30*1000 = 60001 max)
const rawPortOffset = process.env.E2E_PORT_OFFSET;
const E2E_PORT_OFFSET = rawPortOffset === undefined ? process.pid % 30 : Number(rawPortOffset);
if (!Number.isInteger(E2E_PORT_OFFSET) || E2E_PORT_OFFSET < 0 || E2E_PORT_OFFSET > 29) {
throw new Error(`E2E_PORT_OFFSET must be an integer 0-29, got: ${rawPortOffset}`);
}
const BACKEND_BASE_PORT = 18080 + E2E_PORT_OFFSET;
const FRONTEND_BASE_PORT = 13000 + E2E_PORT_OFFSET;
const HEALTH_TIMEOUT_MS = 30_000;
const HEALTH_POLL_MS = 250;
@@ -178,7 +185,7 @@ export const backendFixture = base.extend<object, { backend: BackendContext }>({
// Give each worker its own agentctl port range, offset from the default
// range (10001-10100) to avoid conflicts with a running dev instance.
const agentctlPortBase = 30001 + workerInfo.workerIndex * 50;
const agentctlPortBase = 30001 + E2E_PORT_OFFSET * 1000 + workerInfo.workerIndex * 50;
const agentctlPortMax = agentctlPortBase + 49;
// Install a `git` shim that can sleep on `fetch`/`pull` before execing