diff --git a/.agents/skills/e2e/SKILL.md b/.agents/skills/e2e/SKILL.md index fd92adc4ef..932cad31c0 100644 --- a/.agents/skills/e2e/SKILL.md +++ b/.agents/skills/e2e/SKILL.md @@ -112,8 +112,8 @@ reports a missing `KANDEV_MOCK_AGENT_LINUX_BINARY`, run ```bash make test-e2e # all tests, headless (host) -cd apps && pnpm --filter @kandev/web e2e -- tests/task/my-test.spec.ts # single file -cd apps && pnpm --filter @kandev/web e2e -- --grep "task creation" # by name +cd apps && pnpm --filter @kandev/web e2e:raw -- tests/task/my-test.spec.ts # single file +cd apps && pnpm --filter @kandev/web e2e:raw -- --grep "task creation" # by name ``` ### Flake reproduction @@ -336,7 +336,7 @@ Always verify against the production build before finishing — dev mode can hid playwright-cli close # Kill dev server and backend make build-web -cd apps && pnpm --filter @kandev/web e2e -- tests/path/to/test.spec.ts +cd apps && pnpm --filter @kandev/web e2e:raw -- tests/path/to/test.spec.ts ``` ## Test organization @@ -394,7 +394,7 @@ When a test fails: 3. **Read the failure screenshot** from `e2e/test-results/` — see what the page actually rendered 4. **Attach to the failure** for deeper debugging using `playwright-cli`: ```bash - cd apps && PLAYWRIGHT_HTML_OPEN=never pnpm --filter @kandev/web e2e -- tests/path.spec.ts --debug=cli & + cd apps && PLAYWRIGHT_HTML_OPEN=never pnpm --filter @kandev/web e2e:raw -- tests/path.spec.ts --debug=cli & # Wait for "Debugging Instructions" with session name playwright-cli attach tw- playwright-cli snapshot # inspect page state at failure point diff --git a/.agents/skills/pr-fixup/references/ci-troubleshooting.md b/.agents/skills/pr-fixup/references/ci-troubleshooting.md index dd2ef7a551..23d5bb9fae 100644 --- a/.agents/skills/pr-fixup/references/ci-troubleshooting.md +++ b/.agents/skills/pr-fixup/references/ci-troubleshooting.md @@ -188,8 +188,8 @@ Useful focused commands: ```bash scripts/run-quiet build -- make build-backend build-web -scripts/run-quiet e2e -- bash -c 'cd apps && pnpm --filter @kandev/web e2e -- tests/path/to/failing.spec.ts' -scripts/run-quiet e2e -- bash -c 'cd apps && pnpm --filter @kandev/web e2e -- tests/path/to/failing.spec.ts -g "exact failing test title"' +scripts/run-quiet e2e -- bash -c 'cd apps && pnpm --filter @kandev/web e2e:raw -- tests/path/to/failing.spec.ts' +scripts/run-quiet e2e -- bash -c 'cd apps && pnpm --filter @kandev/web e2e:raw -- tests/path/to/failing.spec.ts -g "exact failing test title"' ``` An isolated pass does not invalidate a CI failure, and a spec outside the PR diff --git a/AGENTS.md b/AGENTS.md index 0bd1f3fb97..2d880c729b 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -20,7 +20,7 @@ apps/ - **Frontend**: Vite/React SPA (`cd apps && pnpm --filter @kandev/web dev|build:vite|lint`; for direct web typecheck use `cd apps/web && pnpm run typecheck`) - **Desktop**: Tauri shell (`cd apps && pnpm --filter @kandev/desktop build|e2e`; Rust tests from `apps/desktop/src-tauri`) - **UI**: Shadcn components via `@kandev/ui` -- **E2E**: Playwright (`cd apps/web && pnpm e2e`). The `containers` project (gated on `KANDEV_E2E_CONTAINERS=1`, formerly `docker`) covers both the Docker executor and the SSH executor — anything that needs a real Docker daemon on the host lives there. See `apps/web/e2e/README.md`. +- **E2E**: Playwright (`cd apps/web && pnpm e2e:raw`). The `containers` project (gated on `KANDEV_E2E_CONTAINERS=1`, formerly `docker`) covers both the Docker executor and the SSH executor — anything that needs a real Docker daemon on the host lives there. See `apps/web/e2e/README.md`. - **GitHub repo**: `https://github.com/kdlbs/kandev` - **Container image**: `ghcr.io/kdlbs/kandev` (GitHub Container Registry) diff --git a/Makefile b/Makefile index 399bffb8a1..7d55fc1aa3 100644 --- a/Makefile +++ b/Makefile @@ -547,8 +547,11 @@ test-scripts: .PHONY: test-e2e test-e2e: build-backend build-web-e2e build-e2e-plugin-package - @printf "$(CYAN)Running E2E tests (headless, parallel)...$(RESET)\n" - @cd $(APPS_DIR) && $(PNPM) --filter @kandev/web e2e + @printf "$(CYAN)Running E2E tests (headless, parallel, managed runner)...$(RESET)\n" + @cd $(WEB_DIR) && status=0; for project in routing auth chromium mobile-chrome containers; do \ + printf "$(CYAN)-- project: $$project --$(RESET)\n"; \ + e2e/scripts/run-e2e.sh --host --no-build --no-strict --shards 1 --project "$$project" -- --output="e2e/test-results-$$project" || status=1; \ + done; exit $$status .PHONY: test-e2e-headed test-e2e-headed: build-backend build-web-e2e build-e2e-plugin-package diff --git a/apps/web/.gitignore b/apps/web/.gitignore index 833d1b46cb..a0b6a47270 100644 --- a/apps/web/.gitignore +++ b/apps/web/.gitignore @@ -12,7 +12,7 @@ # testing /coverage -test-results/ +test-results*/ e2e-artifacts/ playwright-report/ blob-report/ diff --git a/apps/web/e2e/README.md b/apps/web/e2e/README.md index 4a9d7e1638..084528bf93 100644 --- a/apps/web/e2e/README.md +++ b/apps/web/e2e/README.md @@ -27,7 +27,7 @@ fixtures away from tests that count agents or assert the active-agent label. Run it directly with: ```sh -pnpm e2e --project=routing +pnpm e2e:raw --project=routing ``` ### `auth` @@ -37,7 +37,7 @@ backend with authentication enabled, so `chromium` intentionally excludes them. Select the project explicitly; otherwise Playwright reports `No tests found`: ```sh -pnpm e2e --project=auth tests/auth/auth-lifecycle.spec.ts +pnpm e2e:raw --project=auth tests/auth/auth-lifecycle.spec.ts ``` ### `chromium` (default) @@ -45,7 +45,7 @@ pnpm e2e --project=auth tests/auth/auth-lifecycle.spec.ts The everyday surface — runs in every CI shard. Excludes the heavyweight `containers` specs and the mobile specs. ```sh -pnpm e2e +pnpm e2e:raw ``` ### `mobile-chrome` @@ -68,13 +68,13 @@ This project: How to run it locally (requires Docker running): ```sh -KANDEV_E2E_CONTAINERS=1 pnpm e2e --project=containers +KANDEV_E2E_CONTAINERS=1 pnpm e2e:raw --project=containers ``` Or a single spec: ```sh -KANDEV_E2E_CONTAINERS=1 pnpm e2e --project=containers tests/ssh/launch-task.spec.ts +KANDEV_E2E_CONTAINERS=1 pnpm e2e:raw --project=containers tests/ssh/launch-task.spec.ts ``` ### Why "containers" instead of "docker"? @@ -85,18 +85,18 @@ This project used to be named `docker`. It was renamed to `containers` once SSH ## Commands -`e2e`, `e2e:run`, and `e2e:ui` are defined only in `apps/web/package.json`. Run them from `apps/web` (or `pnpm --filter @kandev/web e2e:run` from elsewhere). From the repo root you get `ERR_PNPM_NO_IMPORTER_MANIFEST_FOUND`. +`e2e:raw`, `e2e:run`, and `e2e:ui` are defined only in `apps/web/package.json`. Run them from `apps/web` (or `pnpm --filter @kandev/web e2e:run` from elsewhere). From the repo root you get `ERR_PNPM_NO_IMPORTER_MANIFEST_FOUND`. -| Command | What it does | -| ---------------------------------- | ------------------------------------------------ | -| `pnpm e2e` | Run the default (chromium) project headless. | -| `pnpm e2e:ui` | Open Playwright's UI mode for interactive runs. | -| `pnpm e2e:headed` | Run headless project but with a visible browser. | -| `pnpm e2e --project=containers` | Run container-backed tests (needs Docker). | -| `pnpm e2e --project=mobile-chrome` | Run mobile responsive tests. | -| `pnpm e2e --project=routing` | Run provider-mutating Office routing tests. | -| `pnpm e2e --project=auth` | Run auth-isolated tests. | -| `E2E_DEBUG=1 pnpm e2e` | Surface Docker build output + extra logging. | +| Command | What it does | +| -------------------------------------- | ------------------------------------------------ | +| `pnpm e2e:raw` | Run the default (chromium) project headless. | +| `pnpm e2e:ui` | Open Playwright's UI mode for interactive runs. | +| `pnpm e2e:headed` | Run headless project but with a visible browser. | +| `pnpm e2e:raw --project=containers` | Run container-backed tests (needs Docker). | +| `pnpm e2e:raw --project=mobile-chrome` | Run mobile responsive tests. | +| `pnpm e2e:raw --project=routing` | Run provider-mutating Office routing tests. | +| `pnpm e2e:raw --project=auth` | Run auth-isolated tests. | +| `E2E_DEBUG=1 pnpm e2e:raw` | Surface Docker build output + extra logging. | Common flags: `--shard=1/4`, `-g "fragment of test name"`, `--repeat-each=3` (flake hunting). diff --git a/apps/web/e2e/tests/i18n/pseudo-coverage.spec.ts b/apps/web/e2e/tests/i18n/pseudo-coverage.spec.ts index e7f058df17..c778dee23b 100644 --- a/apps/web/e2e/tests/i18n/pseudo-coverage.spec.ts +++ b/apps/web/e2e/tests/i18n/pseudo-coverage.spec.ts @@ -27,7 +27,7 @@ import { test, expect } from "../../fixtures/test-base"; * the env guard is gone: it now runs unconditionally in the `chromium` project * alongside every other spec. * - * pnpm e2e -- e2e/tests/i18n/pseudo-coverage.spec.ts + * pnpm e2e:raw -- e2e/tests/i18n/pseudo-coverage.spec.ts * * WHAT IT GUARANTEES: on each screen in `SCREENS`, every rendered text node and * every value of a `COPY_ATTRIBUTES` attribute that the detector below can see diff --git a/apps/web/e2e/tests/plugins/plugins-docs-screenshots.spec.ts b/apps/web/e2e/tests/plugins/plugins-docs-screenshots.spec.ts index cc3a3711cb..5f766f0c2c 100644 --- a/apps/web/e2e/tests/plugins/plugins-docs-screenshots.spec.ts +++ b/apps/web/e2e/tests/plugins/plugins-docs-screenshots.spec.ts @@ -18,7 +18,7 @@ * DOCS_PLUGIN_ID=kandev-plugin-hello \ * DOCS_PLUGIN_NAV_ID=hello-world \ * DOCS_PLUGIN_ROUTE=/hello-world \ - * pnpm e2e --project=chromium --workers=1 tests/plugins/plugins-docs-screenshots.spec.ts + * pnpm e2e:raw --project=chromium --workers=1 tests/plugins/plugins-docs-screenshots.spec.ts * * With no DOCS_PLUGIN_* overrides it falls back to the repo's own e2e fixture * package (apps/backend/.build/kandev-plugin-e2e-1.0.0.tar.gz), so the spec is diff --git a/apps/web/e2e/tests/session/session-resume-real.spec.ts b/apps/web/e2e/tests/session/session-resume-real.spec.ts index 4079476f85..b7930f4785 100644 --- a/apps/web/e2e/tests/session/session-resume-real.spec.ts +++ b/apps/web/e2e/tests/session/session-resume-real.spec.ts @@ -12,7 +12,7 @@ import { test } from "../../fixtures/test-base"; * - Backend NOT running with KANDEV_MOCK_AGENT=only * * Run: - * KANDEV_E2E_REAL_AGENTS=1 pnpm --filter @kandev/web e2e -- tests/session-resume-real.spec.ts + * KANDEV_E2E_REAL_AGENTS=1 pnpm e2e:raw -- tests/session-resume-real.spec.ts */ const realAgentsEnabled = process.env.KANDEV_E2E_REAL_AGENTS === "1"; diff --git a/apps/web/package.json b/apps/web/package.json index cad71e7b61..fb13e2f63f 100644 --- a/apps/web/package.json +++ b/apps/web/package.json @@ -16,7 +16,7 @@ "typecheck": "tsc --noEmit", "pretest": "node scripts/generate-release-notes.mjs && node scripts/generate-changelog.mjs", "test": "vitest run", - "e2e": "playwright test --config e2e/playwright.config.ts", + "e2e:raw": "playwright test --config e2e/playwright.config.ts", "e2e:run": "e2e/scripts/run-e2e.sh", "e2e:docker": "e2e/scripts/run-e2e.sh --docker", "e2e:clean": "e2e/scripts/run-e2e.sh clean", diff --git a/docs/public/testing.md b/docs/public/testing.md index f063da1a39..a8a7e367ba 100644 --- a/docs/public/testing.md +++ b/docs/public/testing.md @@ -93,7 +93,7 @@ Projects in `apps/web/e2e/playwright.config.ts` are: | `mobile-chrome` | Pixel 5 responsive browser journeys, not a native mobile app | | `containers` | Real Docker executor and Docker-hosted SSH target; selectable directly and skipped without Docker | -The root `make test-e2e` target leaves project selection to Playwright and therefore runs every project whose prerequisites are available. The managed `pnpm e2e:run` command defaults to `chromium`; select another project explicitly. +The root `make test-e2e` target runs the managed runner once per declared project (`routing`, `auth`, `chromium`, `mobile-chrome`, `containers`), so it covers the same projects Playwright would run unscoped. The managed `pnpm e2e:run` command defaults to `chromium`; select another project explicitly. Run container-backed cases explicitly: diff --git a/docs/test_e2e_web.md b/docs/test_e2e_web.md index 31dac9c268..87c2acae3b 100644 --- a/docs/test_e2e_web.md +++ b/docs/test_e2e_web.md @@ -52,9 +52,9 @@ Or directly via pnpm (requires pre-built binaries and web app): ```bash cd apps -pnpm --filter @kandev/web e2e # All tests -pnpm --filter @kandev/web e2e -- --grep "task creation" # By name -pnpm --filter @kandev/web e2e -- tests/create-task.spec.ts # Single file +pnpm --filter @kandev/web e2e:raw # All tests +pnpm --filter @kandev/web e2e:raw -- --grep "task creation" # By name +pnpm --filter @kandev/web e2e:raw -- tests/create-task.spec.ts # Single file pnpm --filter @kandev/web e2e:headed # With browser pnpm --filter @kandev/web e2e:ui # UI mode ```