José Almeida
39c574bd57
A pass of issues uncovered by running the SSH e2e suite locally
(KANDEV_E2E_CONTAINERS=1 against the real sshd container). After this
commit all 15 non-launch SSH e2e tests pass (UI form validation,
test-connection contract, auth-methods, error-surfacing, sessions-card
empty state) and the launch path runs through prepare -> upload ->
agentctl -> instance -> forward, with WebSocket streams sometimes
disconnecting partway through (tracked below for follow-up).
Image changes (apps/web/e2e/fixtures/ssh-image.ts):
- Install curl so the SSH executor can POST to the in-container
agentctl control server for instance creation.
- Install gcompat + libstdc++ so the dynamically-linked glibc
agentctl binary can exec on musl-libc Alpine. Without it the
kernel returns ENOENT for /lib64/ld-linux-x86-64.so.2.
- Install shadow + 'usermod -p *' kandev so sshd doesn't refuse
pubkey auth for the otherwise-locked user.
- Drop in 'AllowTcpForwarding yes' and 'MaxSessions 100' via
/etc/ssh/sshd_config.d/. Alpine's defaults are 'no' and 10, both
of which crater the SSH executor's port-forward path.
- chown /home/kandev/.kandev to kandev on entrypoint so a host
bind-mount of that path (used in this commit for log persistence)
doesn't make agentctl unable to write its log file.
Image helper (apps/web/e2e/helpers/ssh.ts):
- Bind-mount the host mock-agent-linux-amd64 into the container at
the same absolute path. kandev's orchestrator builds the agent
command from KANDEV_MOCK_AGENT_LINUX_BINARY (a host path), so the
remote needs the binary at that exact path; bake-into-image isn't
enough because the absolute host path is what kandev sends.
- Bind-mount /home/kandev/.kandev so per-session agentctl logs
survive --rm teardown and the test runner can inspect them on
failure.
- Replace the broken CPU-spin sleep helper with a real sleep
subprocess and a /dev/tcp TCP-open probe so timing-dependent
helpers stop racing the container start.
- Retry the host fingerprint scan with stderr capture; the first
scan can race the sshd accept loop on Alpine.
Backend SSH executor (apps/backend/internal/agent/lifecycle):
- executor_ssh_operations.go startRemoteAgentctl: agentctl does not
accept --port-file or --pid-file flags (it reads AGENTCTL_PORT
and writes its own pid). Rewrite the launch wrapper to inject
AGENTCTL_PORT via env, capture $! as the pid, and tail the
on-remote agentctl.log for 'HTTP server bound successfully'
instead of waiting on imaginary side-files.
- executor_ssh_operations.go createRemoteAgentInstance: new helper
that POSTs /api/v1/instances on the in-container control server
via SSH-exec'd curl, returning the per-instance port the executor
should forward to. Mirrors what executor_sprites_operations.go
does inside its sprite. Without this step the agent stream
returns 'configure request failed with status 404' because the
SSH executor was talking to the control server, not the
per-session sub-server.
- executor_ssh.go startAndForwardAgentctl: chains the new
create-instance step in between agentctl launch and port-forward,
and forwards the *instance* port (not the control port).
- executor_ssh.go: drop the per-host SSH connection pool (it was
introducing a subtle aliasing issue between
CreateInstance/StopInstance/GetRemoteStatus that made the SSH
client unreliable across same-host sessions). Each session now
holds its own ssh.Client directly on sshSessionState and
ResumeRemoteInstance dials a fresh client too.
- executor_ssh_operations.go SSHPortForwarder: serialize the
client.Dial calls with a per-forwarder mutex. golang.org/x/crypto/
ssh.Client.Dial is documented as concurrency-safe but in practice
parallel opens from the kandev stream-manager occasionally
produce io.EOF as if the channel-open response was never
received.
Test fix (apps/web/e2e/tests/ssh/error-surfacing.spec.ts):
- Q1 ('TCP refused') was using identity_source='agent' with no
SSH_AUTH_SOCK in the backend env, so it failed at
buildAuthMethods before reaching the TCP-refused path. Switch to
identity_source='file' with the seed key so the test exercises
the failure it claims to.
Known remaining issue (to investigate next): the launch path now
reaches 'agent process started' + 'ACP initialize response
received' when run in isolation, but the WebSocket updates stream
sometimes disconnects with 1006 abnormal closure shortly after. A
standalone Go reproduction of concurrent direct-tcpip channels
against the same image works without issue, so the trigger is
something specific to the kandev backend's stream-manager + my
forwarder interaction. Tracked for the next iteration.
Kandev Web — E2E Test Suite
Playwright-based end-to-end tests. Each Playwright worker spawns its own real Go backend (no mocks of internal services) on isolated ports and a real Next.js standalone frontend, then drives a real Chromium against them.
Project layout
| Folder | What's in it |
|---|---|
tests/ |
Spec files, grouped by feature (chat/, docker/, git/, integrations/, kanban/, pr/, search/, session/, ssh/, layout/, …). |
fixtures/ |
Worker-scoped fixtures that own the backend lifecycle (backend.ts, docker-test-base.ts, ssh-test-base.ts, test-base.ts). |
helpers/ |
Reusable building blocks for specs (api-client.ts, docker.ts, ssh.ts, git-helper.ts, ws-capture.ts, …). |
pages/ |
Page Objects (one class per top-level UI surface — SessionPage, KanbanPage, JiraSettingsPage, SSHSettingsPage, …). |
playwright.config.ts |
Project definitions, timeouts, sharding config. |
global-setup.ts |
Pre-flight checks for required binaries (kandev, mock-agent, standalone next build). |
Playwright projects
The suite is split into three projects. Pick one with --project=<name>.
chromium (default)
The everyday surface — runs in every CI shard. Excludes the heavyweight containers specs and the mobile specs.
pnpm e2e
mobile-chrome
Same as chromium but on Playwright's Pixel-5 viewport, gated on tests/**/mobile-*.spec.ts. Runs in the same CI shard matrix as chromium.
containers — Docker required
This is the "real-infra heavyweight" project. Despite the name, it covers more than just the Docker executor — it's where any test that needs Docker on the host as a runtime lives:
- Docker executor tests (
tests/docker/*.spec.ts) — verify kandev launches realkandev-agent:e2econtainers, recovers them across backend restarts, cleans them up on archive/delete, etc. - SSH executor tests (
tests/ssh/*.spec.ts) — verify kandev SSHes into a realkandev-sshd:e2econtainer, uploads agentctl, runs an agent end-to-end, recovers across backend restarts, etc. The SSH executor's remote target is a Docker container in tests, even though the SSH connection itself is a real SSH connection.
This project:
- Skips entirely when no Docker daemon is reachable. Contributors without Docker can still run
chromium+mobile-chrome. - Builds container images on demand. First run builds
kandev-agent:e2e(slim Node base + git) andkandev-sshd:e2e(Alpine + openssh-server + git + pre-baked mock-agent). Subsequent runs hit Docker's layer cache. - Has a longer per-test timeout (180s vs 60s) because container starts + agent setup are slow.
How to run it locally (requires Docker running):
KANDEV_E2E_CONTAINERS=1 pnpm e2e --project=containers
Or a single spec:
KANDEV_E2E_CONTAINERS=1 pnpm e2e --project=containers tests/ssh/launch-task.spec.ts
Why "containers" instead of "docker"?
This project used to be named docker. It was renamed to containers once SSH e2e tests joined it — calling it docker was misleading because SSH tests have nothing to do with the Docker executor; they just happen to use Docker as the runtime that hosts the sshd target.
KANDEV_E2E_DOCKER=1 is still honored as a deprecated alias for KANDEV_E2E_CONTAINERS=1 for one release. Local scripts and stale CI configs won't break, but new code should use the new name.
Commands
| Command | What it does |
|---|---|
pnpm e2e |
Run the default (chromium) project headless. |
pnpm e2e:ui |
Open Playwright's UI mode for interactive runs. |
pnpm e2e:headed |
Run headless project but with a visible browser. |
pnpm e2e --project=containers |
Run container-backed tests (needs Docker). |
pnpm e2e --project=mobile-chrome |
Run mobile responsive tests. |
E2E_DEBUG=1 pnpm e2e |
Surface Docker build output + extra logging. |
Common flags: --shard=1/4, -g "fragment of test name", --repeat-each=3 (flake hunting).
Backend isolation per worker
Every Playwright worker gets:
- A unique backend port in
BACKEND_BASE_PORT + workerIndex(default18080+). - A unique frontend port in
FRONTEND_BASE_PORT + workerIndex(default13000+). - A fresh tmpdir (
HOME,KANDEV_HOME_DIR, worktree base, repo clone base — all under that tmpdir). - A unique agentctl instance port range (
30001 + E2E_PORT_OFFSET * 1000 + workerIndex * 200). - Its own SQLite DB.
Workers run in parallel across CI shards (--shard=N/M); within a worker, tests run serially because the testPage fixture calls e2eReset on the shared backend before each test.
Mocked vs real
- Mocked: Jira (
KANDEV_MOCK_JIRA=true), Linear (KANDEV_MOCK_LINEAR=true), GitHub (KANDEV_MOCK_GITHUB=true), the agent process itself (KANDEV_MOCK_AGENT=only). These are third-party services or external processes we don't want CI to depend on. - Real: Everything inside the kandev backend — orchestrator, lifecycle manager, agentctl, SSH/SFTP, Docker SDK, git, worktree manager. The point of e2e is to exercise the real code paths.
The SSH executor specifically has no mock controller. Tests use a real Docker-hosted sshd as the remote target, and fault-injection (host-key rotation, dropped traffic, killed pids) is done by operating on the container itself.
Adding a new spec
- Pick a directory under
tests/(or create one for a new feature). - Decide which project it belongs to. Anything that needs Docker →
tests/docker/ortests/ssh/(lands incontainers). Anything mobile-specific → name itmobile-*.spec.ts. Otherwise it joinschromiumautomatically. - Import the right test base:
import { test, expect } from "../../fixtures/test-base";for normal tests.import { test, expect } from "../../fixtures/docker-test-base";for Docker executor tests.import { test, expect } from "../../fixtures/ssh-test-base";for SSH executor tests.
- Use
getByTestIdfor selectors. If the surface you're testing lacks stable testids, add them — drift-prone CSS / text selectors are not worth the maintenance cost.
CI
.github/workflows/e2e-tests.yml defines two jobs:
e2e— matrixedchromium+mobile-chromeshards.e2e-containers— single job, runs--project=containers, needs Docker.
Both upload blob reports that e2e-report merges into a single HTML artifact.