diff --git a/.agents/skills/e2e/SKILL.md b/.agents/skills/e2e/SKILL.md index bfa78dbd6f..2b54a78722 100644 --- a/.agents/skills/e2e/SKILL.md +++ b/.agents/skills/e2e/SKILL.md @@ -63,11 +63,18 @@ Without this, tests run against stale code and failures are misleading. `make bu ## Writing a test 1. Read `helpers/api-client.ts` and `pages/` to discover available seed methods and page objects -2. Import fixtures from `../../fixtures/test-base` — provides `testPage`, `apiClient`, and `seedData` (pre-created workspace with default workflow) +2. Import fixtures from `../../fixtures/test-base` — provides `testPage`, `apiClient`, and `seedData` (pre-created workspace with default workflow). Pull `backend` from the fixture too when you need the backend URL — it's worker-scoped, dynamic, and `process.env.KANDEV_API_BASE_URL` is **not** set in the Playwright runner (only in the frontend SSR child process). Use `backend.baseUrl`. 3. Use `data-testid` attributes for selectors — add them to components as needed 4. Use page objects for common interactions; create new ones for new pages 5. For GitHub features, use `apiClient.mockGitHub*()` methods to seed mock data +### IDs and response shapes — common pitfalls + +- **`apiClient.createTaskWithAgent(...)` returns `CreateTaskResponse`**, which is `Task & { session_id?: string; agent_execution_id?: string }`. Read `created.session_id` directly — don't call `listTaskSessions(taskId)` just to fetch the session that was auto-started by the same call. +- **The URL `/t/:id` contains the TASK ID**, not the session ID. Backend routes like `/port-proxy/:sessionId/:port/*path` expect the session ID. Don't extract IDs from `window.location.pathname` when you need a session ID — pull from the API response. +- **`page.request` shares cookies/storage with the page context**. Fine for the current no-auth local backend; if auth ever lands, this is where you'd plug it in. +- **Preview iframe tests:** the seed repo has no `dev_script` configured, so the preview panel renders a placeholder ("Configure a dev script…") and the URL input never appears — tests that try to drive it hang on the locator timeout. To use the preview iframe in a test, set one first: `await apiClient.updateRepository(seedData.repositoryId, { dev_script: "echo dev" })`. Then click the Preview dockview tab (`await session.clickTab("Preview")`) — the toolbar will mount and the URL input becomes targetable. + Example: ```typescript diff --git a/.agents/skills/pr-fixup/SKILL.md b/.agents/skills/pr-fixup/SKILL.md index 533e3c8c2b..70975a55be 100644 --- a/.agents/skills/pr-fixup/SKILL.md +++ b/.agents/skills/pr-fixup/SKILL.md @@ -160,56 +160,52 @@ Every comment must get a response — either a fix or a reply explaining why it - **Do not mark task 4 completed until every previously-unresolved review thread is either resolved or has an explicit reason documented in a reply.** If you finish the pass and the `isResolved == false` set is still non-empty, you are not done. **Important: issue comments vs review comments use different APIs:** -- **Review comments** (inline, from `gh api repos/:owner/:repo/pulls//comments`) — reply via `/pulls//comments//replies`, react via `/pulls/comments//reactions` -- **Issue comments** (conversation timeline, from `gh pr view --json comments` — e.g., CodeRabbit walkthrough) — reply by posting a new comment via `gh pr comment --body "..."`, react via `/issues/comments//reactions` +- **Review comments** (inline, from `gh api repos/:owner/:repo/pulls//comments`) — use `scripts/pr-resolve` (below). +- **Issue comments** (conversation timeline, from `gh pr view --json comments` — e.g., CodeRabbit walkthrough) — reply by posting a new comment via `gh pr comment --body "..."`, react via `gh api repos/:owner/:repo/issues/comments//reactions -f content="+1"`. There's no "resolve" concept for issue comments. -**For valid comments:** -1. Read the file at the referenced line -2. Implement the fix -3. React with thumbs up: - ```bash - # For review comments: - gh api repos/:owner/:repo/pulls/comments//reactions -f content="+1" - # For issue comments: - gh api repos/:owner/:repo/issues/comments//reactions -f content="+1" - ``` -4. Resolve the review thread (see below for thread ID retrieval) +### Review comments: `scripts/pr-resolve` -**For skipped comments** (already addressed, nitpick, wrong, or outdated): -1. Reply to the comment explaining why it was skipped: - ```bash - # For review comments: - gh api repos/:owner/:repo/pulls//comments//replies -f body="" - # For issue comments: - gh pr comment --body "" - ``` - Examples: - - "This is already handled by X on line Y." - - "This is a style preference not enforced by our linters — keeping as-is." - - "This refers to code that was changed in a later commit." -2. Resolve the review thread +Use the script for every review thread, not just batches — it collapses reply + resolve + +1 reaction into a single call so you don't re-derive the graphql mutation each session. -**Resolving threads:** First fetch thread node IDs to map comment IDs to threads: ```bash -gh api graphql -f query=' -query($owner: String!, $repo: String!, $number: Int!) { - repository(owner: $owner, name: $repo) { - pullRequest(number: $number) { - reviewThreads(first: 100) { - nodes { - id - comments(first: 1) { - nodes { databaseId } - } - } - } - } - } -}' -f owner=":owner" -f repo=":repo" -F number= +# Dump every unresolved thread, TAB-separated (tid, cid, author, path, body_first_120): +scripts/pr-resolve list + +# Reply + resolve + +1 (same call whether you're agreeing or pushing back — +# the body text conveys which): +scripts/pr-resolve reply "Fixed — monotonic counter via useRef. See commit abc1234." +scripts/pr-resolve reply "Acknowledged; the strict source check was relaxed for E2E. Tracking as a follow-up." ``` -Then resolve using the thread `id`: + +For valid comments: read the file, implement the fix, then call `pr-resolve reply` with a body that names the commit or the file:line of the fix. + +For skipped comments (already addressed, nitpick, wrong, outdated): call `pr-resolve reply` with a body that explains why. Examples: +- "This is already handled by X on line Y." +- "This is a style preference not enforced by our linters — keeping as-is." +- "Refers to code that was changed in a later commit." + +For dozens of threads grouped by topic, declare a bash associative array mapping thread IDs → category, then a `reply_for` case that returns the right body per category. Avoids retyping the same explanation across duplicate threads from multiple bots: + ```bash -gh api graphql -f query='mutation { resolveReviewThread(input: {threadId: ""}) { thread { isResolved } } }' +declare -A CAT=( + [PRRT_xxx1]=fixed_counter + [PRRT_xxx2]=fixed_counter + [PRRT_xxx3]=skipped_source_guard +) +declare -A CID=( + [PRRT_xxx1]=3253164429 + [PRRT_xxx2]=3253168996 + [PRRT_xxx3]=3253164669 +) +reply_for() { + case "$1" in + fixed_counter) echo "Fixed — monotonic counter via useRef. See commit abc1234." ;; + skipped_source_guard) echo "Acknowledged; the strict source check was relaxed for E2E. Tracking as a follow-up." ;; + esac +} +for TID in "${!CAT[@]}"; do + scripts/pr-resolve reply "${CID[$TID]}" "$TID" "$(reply_for "${CAT[$TID]}")" +done ``` Mark task 4 as completed. @@ -245,12 +241,6 @@ After the push, CI restarts and bots may re-review. Delegate to `pr-poller` agai - If new review comments appeared after the push → loop back to task 3. - If the poller hit its cap (`recommendation:` mentions "timed out") → surface the remaining pending items to the user and stop. -**Force-push (rebase) can re-open previously resolved inline threads.** GitHub treats the new commit's diff as a fresh review surface, so threads you already replied-to-and-resolved may come back unresolved with their original bodies. Before re-fixing, fetch the comments and check the body — if the issue is already addressed in the new tip, reply "Already fixed in ``" and resolve again rather than re-implementing. - -**Mergeable status lags after a force-push.** `gh pr view --json mergeable` can show `CONFLICTING` / `mergeStateStatus: DIRTY` for ~5-10s while GitHub recomputes against the new tip. Wait briefly before deciding there's a real conflict, and confirm with `git fetch origin main && git diff --name-only HEAD origin/main` locally. - -**Polling that times out is often a "main moved" signal.** A 20-min E2E window is long enough for `main` to ship one or two PRs. Before the next iteration, run `git fetch origin main && git log HEAD..origin/main --oneline` — a non-empty result means a rebase is coming. Rebase, force-push, and re-poll once rather than chasing flaky test reruns. - Cap re-check loops at **3 iterations** to prevent runaway sessions. After 3, surface the remaining state to the user and stop. Mark task 6 as completed. diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000000..ea6f5acdbc --- /dev/null +++ b/.gitattributes @@ -0,0 +1 @@ +scripts/release/update-homebrew-tap.sh linguist-language=Shell diff --git a/.github/workflows/e2e-tests.yml b/.github/workflows/e2e-tests.yml index daf364431a..b61380f0d2 100644 --- a/.github/workflows/e2e-tests.yml +++ b/.github/workflows/e2e-tests.yml @@ -92,7 +92,7 @@ jobs: name: E2E Shard ${{ matrix.shard }}/10 needs: build runs-on: ubuntu-latest - timeout-minutes: 10 + timeout-minutes: 22 strategy: fail-fast: false matrix: diff --git a/.github/workflows/frontend-tests.yml b/.github/workflows/frontend-tests.yml index b8f0735805..3da729bbcb 100644 --- a/.github/workflows/frontend-tests.yml +++ b/.github/workflows/frontend-tests.yml @@ -87,11 +87,23 @@ jobs: exit 0 fi + # Skip symlinks — prettier errors with exit 123 when an explicit + # pattern is a symbolic link. CLAUDE.md files in the tree are symlinks + # to sibling AGENTS.md files (Codex / Claude Code convention). + FILES_TO_CHECK="$( + echo "${CHANGED_FILES}" \ + | sed 's#^apps/##' \ + | while IFS= read -r f; do [ -L "$f" ] || echo "$f"; done + )" + + if [[ -z "${FILES_TO_CHECK}" ]]; then + echo "No non-symlink Prettier-scoped files changed." + exit 0 + fi + echo "Checking Prettier on changed files:" - echo "${CHANGED_FILES}" - echo "${CHANGED_FILES}" \ - | sed 's#^apps/##' \ - | xargs pnpm exec prettier --check --ignore-unknown + echo "${FILES_TO_CHECK}" + echo "${FILES_TO_CHECK}" | xargs pnpm exec prettier --check --ignore-unknown - name: Run lint run: pnpm --filter @kandev/web lint diff --git a/AGENTS.md b/AGENTS.md index 320c8e5182..a944a11cb2 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -6,7 +6,7 @@ ``` apps/ -├── backend/ # Go backend (orchestrator, agent runtime, agentctl, WS gateway) +├── backend/ # Go backend (orchestrator, lifecycle, agentctl, WS gateway) ├── web/ # Next.js frontend (SSR + WS + Zustand) ├── cli/ # CLI tool (TypeScript) ├── landing/ # Landing page @@ -21,7 +21,6 @@ apps/ - **UI**: Shadcn components via `@kandev/ui` - **GitHub repo**: `https://github.com/kdlbs/kandev` - **Container image**: `ghcr.io/kdlbs/kandev` (GitHub Container Registry) -- **Dev mock providers**: In `KANDEV_MOCK_AGENT=true` dev mode the base `mock-agent` is enabled alongside the real CLIs. To exercise the office provider-routing UI without installing real CLIs, set `KANDEV_MOCK_PROVIDERS=claude-acp,codex-acp,opencode-acp` (or any subset of `RoutableProviderIDs`) when launching `make dev` — those canonicals are then replaced with MockAgent aliases. ### Worktrees and commit hooks @@ -29,275 +28,15 @@ The commit-msg hook runs `cd apps && pnpm exec commitlint --edit "$1"`. A fresh --- -## Backend Architecture +## Scoped guidance -### Package Structure +Architecture notes and per-area conventions live alongside the code they describe. Read the file in the directory you're working in: -``` -apps/backend/ -├── cmd/ -│ ├── kandev/ # Main backend binary entry point -│ ├── agentctl/ # Agentctl binary (runs inside containers or standalone) -│ └── mock-agent/ # Mock agent for testing -├── internal/ -│ ├── agent/ -│ │ ├── runtime/ # Agent runtime: single seam for Launch/Resume/Stop/observe -│ │ │ ├── lifecycle/ # Agent instance management (moved from agent/lifecycle) -│ │ │ ├── agentctl/ # HTTP client for talking to agentctl (moved from agentctl/client) -│ │ │ └── routingerr/ # Provider error classifier + sanitizer + ProviderProber registry -│ │ ├── agents/ # Agent type implementations -│ │ ├── controller/ # Agent control operations -│ │ ├── credentials/ # Agent credential management -│ │ ├── discovery/ # Agent discovery -│ │ ├── docker/ # Docker-specific agent logic -│ │ ├── dto/ # Agent data transfer objects -│ │ ├── executor/ # Executor types, checks, and service -│ │ ├── handlers/ # Agent event handlers -│ │ ├── registry/ # Agent type registry and defaults -│ │ ├── settings/ # Agent settings -│ │ ├── mcpconfig/ # MCP server configuration -│ │ └── remoteauth/ # Remote auth catalog and method IDs for remote executors/UI -│ ├── agentctl/ -│ │ └── server/ # agentctl HTTP server -│ │ ├── acp/ # ACP protocol implementation -│ │ ├── adapter/ # Protocol adapters + transport/ (ACP, Codex, OpenCode, Copilot, Amp) -│ │ ├── api/ # HTTP endpoints -│ │ ├── config/ # agentctl configuration -│ │ ├── instance/ # Multi-instance management -│ │ ├── mcp/ # MCP server integration -│ │ ├── process/ # Agent subprocess management -│ │ ├── shell/ # Shell session management -│ │ └── utility/ # agentctl utilities -│ ├── orchestrator/ # Task execution coordination -│ │ ├── dto/ # Orchestrator data transfer objects -│ │ ├── executor/ # Launches agents via lifecycle manager -│ │ ├── handlers/ # Orchestrator event handlers -│ │ ├── messagequeue/ # Message queue for agent prompts -│ │ ├── queue/ # Task queue -│ │ ├── scheduler/ # Task scheduling -│ │ └── watcher/ # Event handlers -│ ├── task/ -│ │ ├── controller/ # Task HTTP/WS controllers -│ │ ├── dto/ # Task data transfer objects -│ │ ├── events/ # Task event types -│ │ ├── handlers/ # Task event handlers -│ │ ├── models/ # Task, Session, Executor, Message models -│ │ ├── repository/ # Database access (SQLite) -│ │ ├── service/ # Task business logic -│ │ └── share/ # Public share links: snapshot builder, gist backend, service, redaction -│ ├── office/ # Autonomous agent management (Office feature) -│ │ ├── agents/ # Agent instance CRUD + auth guards -│ │ ├── approvals/ # Approval requests and decisions -│ │ ├── channels/ # External integration channels (webhooks) -│ │ ├── config/ # Config sync (DB ↔ filesystem) -│ │ ├── configloader/ # Filesystem config reader/writer -│ │ ├── costs/ # Cost tracking and budget policies -│ │ ├── dashboard/ # Dashboard API, issues, activity, live runs -│ │ ├── infra/ # GC, reconciliation -│ │ ├── labels/ # Task labels -│ │ ├── onboarding/ # Workspace onboarding wizard API -│ │ ├── projects/ # Project management -│ │ ├── repository/ # Office SQLite persistence -│ │ ├── runtime/ # Agent run context, capabilities, and runtime action surface -│ │ ├── routines/ # Scheduled recurring tasks -│ │ ├── routing/ # Provider routing: resolver, validators, catalogue, backoff, agent-overrides -│ │ ├── scheduler/ # Wakeup scheduler (duplicate of service scheduler features) -│ │ ├── service/ # Core office service (wakeups, event subscribers, execution policy) -│ │ ├── shared/ # Shared interfaces and activity logging -│ │ ├── skills/ # Skill injection and materialization -│ │ └── workspaces/ # Workspace deletion handler -│ ├── events/ # Event bus for internal pub/sub -│ ├── gateway/ # WebSocket gateway -│ ├── github/ # GitHub API integration (PRs, reviews, webhooks) -│ ├── gitlab/ # GitLab API integration (MRs, discussions, pipelines) -│ ├── common/ # Shared utilities, config, logger -│ ├── integration/ # External integrations -│ ├── integrations/ # Shared shapes for third-party integrations -│ │ ├── healthpoll/ # Reusable 90s auth-health Poller (used by jira, linear) -│ │ └── secretadapter/ # Upsert-style adapter over secrets.SecretStore -│ ├── jira/ # Jira/Atlassian Cloud integration (config, REST client, poller) -│ ├── linear/ # Linear integration (config, GraphQL client, poller) -│ ├── lsp/ # LSP server -│ ├── mcp/ # MCP protocol support -│ ├── health/ # Health check endpoints -│ ├── notifications/ # Notification system -│ ├── persistence/ # Persistence layer -│ ├── prompts/ # Prompt management -│ ├── repoclone/ # Repository cloning for remote executors -│ ├── scriptengine/ # Script placeholder resolution and interpolation -│ ├── secrets/ # Secret management -│ ├── sprites/ # Sprites AI integration -│ ├── sysprompt/ # System prompt injection -│ ├── task/ -│ │ └── ... -│ ├── tools/ # Tool integrations -│ ├── user/ # User management -│ ├── utility/ # Shared utility functions -│ ├── workflow/ # Workflow engine -│ │ ├── engine/ # Typed state-machine engine -│ │ ├── models/ # Workflow step, template, and history models -│ │ ├── repository/ # Workflow persistence (SQLite) -│ │ └── service/ # Workflow CRUD and step resolution -│ └── worktree/ # Git worktree management for workspace isolation -``` - -### Key Concepts - -**Orchestrator** coordinates task execution: -- Receives task start/stop/resume requests via WebSocket -- Delegates to lifecycle manager for agent operations -- Handles event-driven state transitions via workflow engine -- Located in `internal/orchestrator/` - -**Workflow Engine** (`internal/workflow/engine/`) provides typed state-machine evaluation: -- `Engine.HandleTrigger()` evaluates step actions for triggers (on_enter, on_turn_start, on_turn_complete, on_exit) -- `TransitionStore` interface abstracts persistence (implemented by `orchestrator.workflowStore`) -- `CallbackRegistry` maps action kinds to callbacks (plan mode, auto-start, context reset) -- First-transition-wins: multiple transition actions in one trigger, first eligible wins -- `EvaluateOnly` mode: engine evaluates without persisting, caller orchestrates on_exit → DB → on_enter -- `RequiresApproval` on actions: transitions requiring review gating are skipped -- Idempotent by `OperationID`; session-scoped data bag via `MachineState.Data` - -**Agent Runtime** (`internal/agent/runtime/`) is the single seam for launching, resuming, stopping, and observing agent executions. ADR 0004 introduced this in Phase 1 of task-model-unification. The public surface is `runtime.Runtime` (`runtime.go`); a thin facade (`facade.go`) delegates to a `Backend` (satisfied by `*lifecycle.Manager`). - -**Convention:** only `internal/agent/runtime/` (and code that pre-dates Phase 1 migration) may import `runtime/lifecycle` or `runtime/agentctl` directly. New consumers — workflow engine actions, cron-driven trigger handlers, future task-tier callers — should depend on `runtime.Runtime`. Existing call sites are migrated through later phases of task-model-unification. - -**Lifecycle Manager** (`internal/agent/runtime/lifecycle/`) manages agent instances under the runtime: -- `Manager` (`manager.go`, `manager_*.go`) - central coordinator for agent lifecycle -- `ExecutorBackend` interface (`executor_backend.go`) - abstracts execution environment (Docker, Standalone, Sprites, Remote Docker) -- `ExecutionStore` (`execution_store.go`) - thread-safe in-memory execution tracking -- `session.go` - ACP session initialization and resume -- `streams.go` - WebSocket stream connections to agentctl -- `process_runner.go` - agent process launch and management -- `profile_resolver.go` - resolves agent profiles/settings - -**agentctl client** (`internal/agent/runtime/agentctl/`) is the HTTP/WS client used by the lifecycle manager to talk to a running agentctl instance. It is a runtime-tier package and should not be imported outside `internal/agent/runtime/`. - -**agentctl** is an HTTP server that: -- Runs inside Docker containers or as standalone process -- Manages agent subprocess via stdin/stdout (ACP protocol) -- Exposes workspace operations (shell, git, files) -- Supports multiple concurrent instances on different ports - -**Executor Types** (database model): -- `local_pc` - Standalone process on host ✅ -- `local_docker` - Docker container on host ✅ -- `sprites` - Sprites cloud environment ✅ -- `remote_docker`, `remote_vps`, `k8s` - Planned - -### Execution Flow - -``` -Client (WS) → Orchestrator → Lifecycle Manager → ExecutorBackend (container/process) → agentctl - ↓ -Client (WS) ← Orchestrator ← Lifecycle Manager ←──── stream updates (WS) ──────── agent subprocess -``` - -1. Orchestrator receives `session.launch` via WS -2. Lifecycle Manager creates executor instance (container or process) -3. agentctl starts inside the instance, agent subprocess is configured and started -4. Agent events stream back via WS through the chain - -**Session Resume:** `TaskSession.ACPSessionID` stored for resume; `ExecutorRunning` tracks active state; on restart `RecoverInstances()` reconnects. - -**Provider Pattern:** Packages expose `Provide(cfg, log) (*impl, cleanup, error)` for DI. Returns implementation, cleanup function, and error. Cleanup called during graceful shutdown. - -**Worktrees:** `internal/worktree/Manager` provides workspace isolation. Each session can have its own worktree (branch) to prevent conflicts between concurrent agents. - -**Executor default scripts:** Default prepare scripts are in `internal/agent/runtime/lifecycle/default_scripts.go`; `internal/scriptengine/` handles placeholder resolution. - ---- - -## agentctl Server - -### API Groups - -agentctl exposes these route groups (see `internal/agentctl/server/api/`): -- `/health`, `/info`, `/status` - Health and status -- `/instances/*` - Multi-instance management -- `/processes/*` - Agent subprocess management (start/stop) -- `/agent/configure`, `/agent/stream` - Agent configuration and event streaming -- `/git/*` - Git operations (status, commit, push, pull, rebase, stage, etc.) -- `/shell/*` - Shell session management -- `/workspace/*` - File operations, search, tree -- `/vscode/*` - VS Code integration proxy - -### Adapter Model - -Protocol adapters in `adapter/transport/` normalize different agent CLIs: -- `AgentAdapter` interface defines `Start()`, `Stop()`, `Prompt()`, `Cancel()` -- Transports: `acp` (Claude Code), `codex` (OpenAI Codex), `opencode`, `shared`, `streamjson` -- Top-level adapters: `CopilotAdapter` (GitHub Copilot SDK), `AmpAdapter` (Sourcegraph Amp) -- `process.Manager` owns subprocess, wires stdio to adapter -- Factory pattern in `adapter/factory.go` selects adapter by agent type - -### ACP Protocol - -JSON-RPC 2.0 over stdin/stdout between agentctl and agent process. Requests: `initialize`, `session/new`, `session/load`, `session/prompt`, `session/cancel`. Notifications: `session/update` with types `message_chunk`, `tool_call`, `tool_update`, `complete`, `error`, `permission_request`, `context_window`. - ---- - -## Frontend Architecture - -### UI Components - -**Shadcn Components:** Import from `@kandev/ui` package: -```typescript -import { Badge } from '@kandev/ui/badge'; -import { Button } from '@kandev/ui/button'; -import { Dialog } from '@kandev/ui/dialog'; -// etc... -``` - -**Do NOT** import from `@/components/ui/*` - always use `@kandev/ui` package. - -### Data Flow Pattern (Critical) - -``` -SSR Fetch -> Hydrate Store -> Components Read Store -> Hooks Subscribe -``` - -**Never fetch data directly in components.** - -### Store Structure (Domain Slices) - -``` -lib/state/ -├── store.ts # Root composition -├── default-state.ts # Default state + initial state merge -├── slices/ # Domain slices -│ ├── kanban/ # boards, tasks, columns -│ ├── session/ # sessions, messages, turns, worktrees -│ ├── session-runtime/ # shell, processes, git, context -│ ├── workspace/ # workspaces, repos, branches -│ ├── settings/ # executors, agents, editors, prompts (incl. userSettings) -│ ├── office/ # office agents, skills, projects, dashboard, issues -│ ├── comments/ # code review diff comments -│ ├── github/ # GitHub PRs, reviews -│ └── ui/ # preview, connection, active state, sidebar views -├── hydration/ # SSR merge strategies - -hooks/domains/{kanban,session,workspace,settings,comments,github}/ # Domain-organized hooks -lib/api/domains/ # API clients -├── kanban-api, session-api, workspace-api, settings-api, process-api -├── plan-api, queue-api, workflow-api, stats-api, github-api -├── office-api, office-extended-api, tree-api -├── user-shell-api, debug-api, secrets-api, sprites-api, vscode-api -├── health-api, utility-api -``` - -**Key State Paths:** -- `messages.bySession[sessionId]`, `shell.outputs[sessionId]`, `gitStatus.bySessionId[sessionId]` -- `tasks.activeTaskId`, `tasks.activeSessionId`, `workspaces.activeId` -- `repositories.byWorkspace`, `repositoryBranches.byRepository` - -**Hydration:** `lib/state/hydration/merge-strategies.ts` has `deepMerge()`, `mergeSessionMap()`, `mergeLoadingState()` to avoid overwriting live client state. Pass `activeSessionId` to protect active sessions. - -**Hooks Pattern:** Hooks in `hooks/domains/` encapsulate WS subscription + store selection. WS client deduplicates subscriptions automatically. - -### WS - -**Format:** `{id, type, action, payload, timestamp}`. +- `apps/backend/AGENTS.md` — Go backend: package structure (incl. `internal/office/` and `internal/agent/runtime/`), key concepts (orchestrator, workflow engine, agent runtime, lifecycle manager), execution flow, provider pattern, backups, testing conventions, Go lint limits. +- `apps/backend/internal/agentctl/AGENTS.md` — agentctl HTTP server: route groups, adapter model, ACP protocol. +- `apps/backend/internal/agentctl/server/api/AGENTS.md` — reverse-proxy body rewriting (`Accept-Encoding`), iframe-blocking header stripping. +- `apps/backend/internal/integrations/AGENTS.md` — adding a new third-party integration (Jira/Linear pattern, both backend and frontend halves). The `/add-integration` skill mirrors this for scaffolding new integrations. +- `apps/web/AGENTS.md` — Next.js frontend: shadcn imports, SSR-hydrate-store data flow, store slice structure (incl. `office`), WS format, component conventions, TS lint limits. --- @@ -305,30 +44,22 @@ lib/api/domains/ # API clients ### Commit Conventions (enforced by CI) -Commits to `main` **must** follow [Conventional Commits](https://www.conventionalcommits.org/) (`type: description`). PRs are squash-merged — the PR title becomes the commit, validated by CI. Changelog is auto-generated from these via git-cliff (`cliff.toml`). See `.agents/skills/commit/SKILL.md` for allowed types and examples. +Commits to `main` **must** follow [Conventional Commits](https://www.conventionalcommits.org/) (`type: description`). PRs are squash-merged - the PR title becomes the commit, validated by CI. Changelog is auto-generated from these via git-cliff (`cliff.toml`). See `.agents/skills/commit/SKILL.md` for allowed types and examples. + +The commitlint hook caps the header at **100 characters** (`type(scope): description`). The pre-commit prettier hook also reformats staged TS/TSX files - if it does, the commit fails. Re-stage the reformatted files and create a new commit (don't `--amend`). ### Release & Versioning Kandev uses a **single SemVer** `X.Y.Z` across npm, Homebrew, and GitHub release; release flow runs entirely in CI via `.github/workflows/release.yml`. Full details in the `/release` skill — load it when cutting a release or debugging release artifacts. -### Code Quality (enforced by linters) +### Code Quality -Static analysis runs in CI and pre-commit. New code **must** stay within these limits: +Static analysis runs in CI and pre-commit. Each subtree has its own thresholds: -**Go** (`apps/backend/.golangci.yml` - errors on new code only): -- Functions: **≤80 lines**, **≤50 statements** -- Cyclomatic complexity: **≤15** · Cognitive complexity: **≤30** -- Nesting depth: **≤5** · Naked returns only in functions **≤30 lines** -- No duplicated blocks (**≥150 tokens**) · Repeated strings → constants (**≥3 occurrences**) +- Go limits: see `apps/backend/AGENTS.md` (and `apps/backend/.golangci.yml`). +- TypeScript limits: see `apps/web/AGENTS.md` (and `apps/web/eslint.config.mjs`). -**TypeScript** (`apps/web/eslint.config.mjs` - warnings, will become errors): -- Files: **≤600 lines** · Functions: **≤100 lines** -- Cyclomatic complexity: **≤15** · Cognitive complexity: **≤20** -- Nesting depth: **≤4** · Parameters: **≤5** -- No duplicated strings (**≥4 occurrences**) · No identical functions · No unused imports -- No nested ternaries - -**When you hit a limit:** extract a helper function, custom hook, or sub-component. Prefer composition over growing a single function. +When you hit a limit: extract a helper function, custom hook, or sub-component. Prefer composition over growing a single function. ### Testing @@ -339,34 +70,6 @@ Every code change must include tests for new or changed logic. Backend: `*_test. - **Decisions:** Architecture decisions are recorded in `docs/decisions/`. Read `docs/decisions/INDEX.md` for an overview. When making significant architectural choices, create a new ADR via `/record decision`. - **Plans:** Implementation plans are generated from specs via `/plan` and saved to `docs/specs//plan.md`. Plans are gitignored (working files, not committed) — only specs and ADRs are tracked. -### Backend -- Provider pattern for DI; stderr for logs, stdout for ACP only -- Pass context through chains; event bus for cross-component comm -- **Execution access:** Workspace-oriented handlers (files, shell, inference, ports, vscode, LSP) MUST use `GetOrEnsureExecution(ctx, sessionID)` — it recovers from backend restarts by creating executions on-demand. Only use `GetExecutionBySessionID` for operations that require a running agent process (prompt, cancel, mode). -- **Task lifecycle events:** Any code path that mutates a task row must publish via the event bus (`task.created` / `task.updated` / `task.deleted`) — either by going through `Service.CreateTask` / `UpdateTask` / `DeleteTask` / `ArchiveTask`, or by calling `publishTaskEvent` (or one of the `Publish*` helpers in `service_events.go`) directly. Walking `repository.TaskRepository` straight bypasses event publishing and breaks WS-driven UI like the All-Workflows kanban view. `HandoffService`'s cascade methods learned this the hard way — they now require a `TaskEventPublisher` wired via `SetTaskEventPublisher`. New cascade / bulk / cleanup paths must follow the same pattern. -- **Testing:** Prefer `testing/synctest` (Go 1.24+) over `time.Sleep` for time-dependent tests. Use `synctest.Test` to wrap tests with tickers or timeouts — it advances fake time instantly when all goroutines are idle. When `synctest` is not feasible (e.g., tests spawning external processes like `git`), use channel-based synchronization (`<-started`, non-blocking `select`) instead of sleep-based waits. Reserve `time.Sleep` only for integration tests that need real subprocess execution time. - -### Backups -- On every SQLite boot, `persistence.Provide` reads `kandev_meta.kandev_version`. If the stored version differs from the binary version (or any user tables exist but no version is recorded), it takes a `VACUUM INTO` snapshot into `/backups/` before running migrations. -- Retention: 2 backups kept (newest two by mtime); older ones are pruned after the snapshot succeeds. -- Postgres: backup is skipped with a log line. Use `pg_dump` for Postgres backups. -- Boot aborts if the backup fails -- the pool is closed and `Provide` returns an error. -- After all repos complete `initSchema`, `cmd/kandev/storage.go:recordSchemaVersion` writes the current binary version into `kandev_meta` (non-fatal; a failure just means the next boot will take a fresh snapshot). -- Migration logging: `db.MigrateLogger.Apply(name, stmt)` -- success logs Info, "already exists"/"duplicate column name" is silently swallowed, anything else logs Warn but never returns an error (preserving the existing swallow-error contract). - -### Frontend -- **Data:** SSR fetch → hydrate → read store. Never fetch in components -- **UI Components:** - - Import shadcn components from `@kandev/ui`, NOT `@/components/ui/*` - - **Always prefer native shadcn components** over custom implementations - - Check `apps/packages/ui/src/` for available components (pagination, table, dialog, etc.) - - For data tables, use `@kandev/ui/table` with TanStack Table; use shadcn Pagination components - - Only create custom components when shadcn doesn't provide what's needed -- **Components:** <200 lines, extract to domain components, composition over props -- **Hooks:** Domain-organized in `hooks/domains/`, encapsulate subscription + selection -- **WS:** Use subscription hooks only; client auto-deduplicates -- **Interactivity:** All buttons and links with actions must have `cursor-pointer` class - ### Plan Implementation - After implementing a plan, run `make fmt` first to format code, then run `make typecheck test lint` to verify the changes. Formatting must come first because formatters may split lines, which can trigger complexity linter warnings. @@ -378,7 +81,7 @@ Skills use `gh` CLI by default. If a `gh` command fails (not installed, not auth ### Third-party integrations -Jira and Linear are the model (per-workspace credentials, 90s auth-health poller via `internal/integrations/healthpoll`, settings page with status banner). New integrations should **reuse the shared shapes** rather than copying either. Full layout, file conventions, and Jira-vs-Linear divergence notes in the `/add-integration` skill — load it when scaffolding a new integration. +Jira and Linear are the model (per-workspace credentials, 90s auth-health poller via `internal/integrations/healthpoll`, settings page with status banner). New integrations should **reuse the shared shapes** rather than copying either. Full layout, file conventions, and Jira-vs-Linear divergence notes in `apps/backend/internal/integrations/AGENTS.md` and the `/add-integration` skill — load either when scaffolding a new integration. ### Runtime profiles (prod / dev / e2e) @@ -394,6 +97,8 @@ To flip a feature on for every user: change its `prod:` to `"true"` in `profiles This file is read by AI coding agents (Claude Code via `CLAUDE.md` symlink, Codex via `AGENTS.md`). If your changes make any section of this file outdated or inaccurate - e.g., you add/remove/rename packages, change architectural patterns, add new adapters, modify store slices, or change conventions - **update the relevant sections of this file as part of the same PR**. Keep descriptions concise and factual. Do not add speculative or aspirational content. +When a change is scoped to a single subtree, update the scoped `AGENTS.md` instead of (or in addition to) this root file. See the "Scoped guidance" pointers at the top. + --- **Last Updated**: 2026-05-16 diff --git a/apps/backend/AGENTS.md b/apps/backend/AGENTS.md new file mode 100644 index 0000000000..a449d90952 --- /dev/null +++ b/apps/backend/AGENTS.md @@ -0,0 +1,207 @@ +# Backend (Go) — architecture and conventions + +Scoped guidance for `apps/backend/`. Repo-wide rules (commit format, code-quality limits, etc.) live in the root `AGENTS.md`. + +## Package Structure + +```text +apps/backend/ +├── cmd/ +│ ├── kandev/ # Main backend binary entry point +│ ├── agentctl/ # Agentctl binary (runs inside containers or standalone) +│ └── mock-agent/ # Mock agent for testing +├── internal/ +│ ├── agent/ +│ │ ├── runtime/ # Agent runtime: single seam for Launch/Resume/Stop/observe +│ │ │ ├── lifecycle/ # Agent instance management (moved from agent/lifecycle) +│ │ │ ├── agentctl/ # HTTP client for talking to agentctl (moved from agentctl/client) +│ │ │ └── routingerr/ # Provider error classifier + sanitizer + ProviderProber registry +│ │ ├── agents/ # Agent type implementations +│ │ ├── controller/ # Agent control operations +│ │ ├── credentials/ # Agent credential management +│ │ ├── discovery/ # Agent discovery +│ │ ├── docker/ # Docker-specific agent logic +│ │ ├── dto/ # Agent data transfer objects +│ │ ├── executor/ # Executor types, checks, and service +│ │ ├── handlers/ # Agent event handlers +│ │ ├── registry/ # Agent type registry and defaults +│ │ ├── settings/ # Agent settings +│ │ ├── mcpconfig/ # MCP server configuration +│ │ └── remoteauth/ # Remote auth catalog and method IDs for remote executors/UI +│ ├── agentctl/ +│ │ └── server/ # agentctl HTTP server +│ │ ├── acp/ # ACP protocol implementation +│ │ ├── adapter/ # Protocol adapters + transport/ (ACP, Codex, OpenCode, Copilot, Amp) +│ │ ├── api/ # HTTP endpoints +│ │ ├── config/ # agentctl configuration +│ │ ├── instance/ # Multi-instance management +│ │ ├── mcp/ # MCP server integration +│ │ ├── process/ # Agent subprocess management +│ │ ├── shell/ # Shell session management +│ │ └── utility/ # agentctl utilities +│ ├── orchestrator/ # Task execution coordination +│ │ ├── dto/ # Orchestrator data transfer objects +│ │ ├── executor/ # Launches agents via lifecycle manager +│ │ ├── handlers/ # Orchestrator event handlers +│ │ ├── messagequeue/ # Message queue for agent prompts +│ │ ├── queue/ # Task queue +│ │ ├── scheduler/ # Task scheduling +│ │ └── watcher/ # Event handlers +│ ├── task/ +│ │ ├── controller/ # Task HTTP/WS controllers +│ │ ├── dto/ # Task data transfer objects +│ │ ├── events/ # Task event types +│ │ ├── handlers/ # Task event handlers +│ │ ├── models/ # Task, Session, Executor, Message models +│ │ ├── repository/ # Database access (SQLite) +│ │ └── service/ # Task business logic +│ ├── office/ # Autonomous agent management (Office feature) +│ │ ├── agents/ # Agent instance CRUD + auth guards +│ │ ├── approvals/ # Approval requests and decisions +│ │ ├── channels/ # External integration channels (webhooks) +│ │ ├── config/ # Config sync (DB ↔ filesystem) +│ │ ├── configloader/ # Filesystem config reader/writer +│ │ ├── costs/ # Cost tracking and budget policies +│ │ ├── dashboard/ # Dashboard API, issues, activity, live runs +│ │ ├── infra/ # GC, reconciliation +│ │ ├── labels/ # Task labels +│ │ ├── onboarding/ # Workspace onboarding wizard API +│ │ ├── projects/ # Project management +│ │ ├── repository/ # Office SQLite persistence +│ │ ├── runtime/ # Agent run context, capabilities, and runtime action surface +│ │ ├── routines/ # Scheduled recurring tasks +│ │ ├── routing/ # Provider routing: resolver, validators, catalogue, backoff, agent-overrides +│ │ ├── scheduler/ # Wakeup scheduler (duplicate of service scheduler features) +│ │ ├── service/ # Core office service (wakeups, event subscribers, execution policy) +│ │ ├── shared/ # Shared interfaces and activity logging +│ │ ├── skills/ # Skill injection and materialization +│ │ └── workspaces/ # Workspace deletion handler +│ ├── events/ # Event bus for internal pub/sub +│ ├── gateway/ # WebSocket gateway +│ ├── github/ # GitHub API integration (PRs, reviews, webhooks) +│ ├── common/ # Shared utilities, config, logger +│ ├── integration/ # External integrations +│ ├── integrations/ # Shared shapes for third-party integrations +│ │ ├── healthpoll/ # Reusable 90s auth-health Poller (used by jira, linear) +│ │ └── secretadapter/ # Upsert-style adapter over secrets.SecretStore +│ ├── jira/ # Jira/Atlassian Cloud integration (config, REST client, poller) +│ ├── linear/ # Linear integration (config, GraphQL client, poller) +│ ├── lsp/ # LSP server +│ ├── mcp/ # MCP protocol support +│ ├── health/ # Health check endpoints +│ ├── notifications/ # Notification system +│ ├── persistence/ # Persistence layer +│ ├── prompts/ # Prompt management +│ ├── repoclone/ # Repository cloning for remote executors +│ ├── scriptengine/ # Script placeholder resolution and interpolation +│ ├── secrets/ # Secret management +│ ├── sprites/ # Sprites AI integration +│ ├── sysprompt/ # System prompt injection +│ ├── tools/ # Tool integrations +│ ├── user/ # User management +│ ├── utility/ # Shared utility functions +│ ├── workflow/ # Workflow engine +│ │ ├── engine/ # Typed state-machine engine +│ │ ├── models/ # Workflow step, template, and history models +│ │ ├── repository/ # Workflow persistence (SQLite) +│ │ └── service/ # Workflow CRUD and step resolution +│ └── worktree/ # Git worktree management for workspace isolation +``` + +## Key Concepts + +**Orchestrator** coordinates task execution: +- Receives task start/stop/resume requests via WebSocket +- Delegates to lifecycle manager for agent operations +- Handles event-driven state transitions via workflow engine +- Located in `internal/orchestrator/` + +**Workflow Engine** (`internal/workflow/engine/`) provides typed state-machine evaluation: +- `Engine.HandleTrigger()` evaluates step actions for triggers (on_enter, on_turn_start, on_turn_complete, on_exit) +- `TransitionStore` interface abstracts persistence (implemented by `orchestrator.workflowStore`) +- `CallbackRegistry` maps action kinds to callbacks (plan mode, auto-start, context reset) +- First-transition-wins: multiple transition actions in one trigger, first eligible wins +- `EvaluateOnly` mode: engine evaluates without persisting, caller orchestrates on_exit → DB → on_enter +- `RequiresApproval` on actions: transitions requiring review gating are skipped +- Idempotent by `OperationID`; session-scoped data bag via `MachineState.Data` + +**Agent Runtime** (`internal/agent/runtime/`) is the single seam for launching, resuming, stopping, and observing agent executions. ADR 0004 introduced this in Phase 1 of task-model-unification. The public surface is `runtime.Runtime` (`runtime.go`); a thin facade (`facade.go`) delegates to a `Backend` (satisfied by `*lifecycle.Manager`). + +**Convention:** only `internal/agent/runtime/` (and code that pre-dates Phase 1 migration) may import `runtime/lifecycle` or `runtime/agentctl` directly. New consumers — workflow engine actions, cron-driven trigger handlers, future task-tier callers — should depend on `runtime.Runtime`. Existing call sites are migrated through later phases of task-model-unification. + +**Lifecycle Manager** (`internal/agent/runtime/lifecycle/`) manages agent instances under the runtime: +- `Manager` (`manager.go`, `manager_*.go`) - central coordinator for agent lifecycle +- `ExecutorBackend` interface (`executor_backend.go`) - abstracts execution environment (Docker, Standalone, Sprites, Remote Docker) +- `ExecutionStore` (`execution_store.go`) - thread-safe in-memory execution tracking +- `session.go` - ACP session initialization and resume +- `streams.go` - WebSocket stream connections to agentctl +- `process_runner.go` - agent process launch and management +- `profile_resolver.go` - resolves agent profiles/settings + +**agentctl client** (`internal/agent/runtime/agentctl/`) is the HTTP/WS client used by the lifecycle manager to talk to a running agentctl instance. It is a runtime-tier package and should not be imported outside `internal/agent/runtime/`. + +**agentctl** is an HTTP server that: +- Runs inside Docker containers or as standalone process +- Manages agent subprocess via stdin/stdout (ACP protocol) +- Exposes workspace operations (shell, git, files) +- Supports multiple concurrent instances on different ports + +**Executor Types** (database model): +- `local_pc` - Standalone process on host +- `local_docker` - Docker container on host +- `sprites` - Sprites cloud environment +- `remote_docker`, `remote_vps`, `k8s` - Planned + +## Execution Flow + +```text +Client (WS) → Orchestrator → Lifecycle Manager → ExecutorBackend (container/process) → agentctl + ↓ +Client (WS) ← Orchestrator ← Lifecycle Manager ←──── stream updates (WS) ──────── agent subprocess +``` + +1. Orchestrator receives `session.launch` via WS +2. Lifecycle Manager creates executor instance (container or process) +3. agentctl starts inside the instance, agent subprocess is configured and started +4. Agent events stream back via WS through the chain + +**Session Resume:** `TaskSession.ACPSessionID` stored for resume; `ExecutorRunning` tracks active state; on restart `RecoverInstances()` reconnects. + +**Provider Pattern:** Packages expose `Provide(cfg, log) (*impl, cleanup, error)` for DI. Returns implementation, cleanup function, and error. Cleanup called during graceful shutdown. + +**Worktrees:** `internal/worktree/Manager` provides workspace isolation. Each session can have its own worktree (branch) to prevent conflicts between concurrent agents. + +**Executor default scripts:** Default prepare scripts are in `internal/agent/runtime/lifecycle/default_scripts.go`; `internal/scriptengine/` handles placeholder resolution. + +## Conventions + +- Provider pattern for DI; stderr for logs, stdout for ACP only. +- Pass context through chains; event bus for cross-component comm. +- **Execution access:** Workspace-oriented handlers (files, shell, inference, ports, vscode, LSP) MUST use `GetOrEnsureExecution(ctx, sessionID)` — it recovers from backend restarts by creating executions on-demand. Only use `GetExecutionBySessionID` for operations that require a running agent process (prompt, cancel, mode). +- **Task lifecycle events:** Any code path that mutates a task row must publish via the event bus (`task.created` / `task.updated` / `task.deleted`) — either by going through `Service.CreateTask` / `UpdateTask` / `DeleteTask` / `ArchiveTask`, or by calling `publishTaskEvent` (or one of the `Publish*` helpers in `service_events.go`) directly. Walking `repository.TaskRepository` straight bypasses event publishing and breaks WS-driven UI like the All-Workflows kanban view. `HandoffService`'s cascade methods learned this the hard way — they now require a `TaskEventPublisher` wired via `SetTaskEventPublisher`. New cascade / bulk / cleanup paths must follow the same pattern. +- **Testing:** Prefer `testing/synctest` (Go 1.24+) over `time.Sleep` for time-dependent tests. Use `synctest.Test` to wrap tests with tickers or timeouts — it advances fake time instantly when all goroutines are idle. When `synctest` is not feasible (e.g., tests spawning external processes like `git`), use channel-based synchronization (`<-started`, non-blocking `select`) instead of sleep-based waits. Reserve `time.Sleep` only for integration tests that need real subprocess execution time. + +## Backups + +- On every SQLite boot, `persistence.Provide` reads `kandev_meta.kandev_version`. If the stored version differs from the binary version (or any user tables exist but no version is recorded), it takes a `VACUUM INTO` snapshot into `/backups/` before running migrations. +- Retention: 2 backups kept (newest two by mtime); older ones are pruned after the snapshot succeeds. +- Postgres: backup is skipped with a log line. Use `pg_dump` for Postgres backups. +- Boot aborts if the backup fails — the pool is closed and `Provide` returns an error. +- After all repos complete `initSchema`, `cmd/kandev/storage.go:recordSchemaVersion` writes the current binary version into `kandev_meta` (non-fatal; a failure just means the next boot will take a fresh snapshot). +- Migration logging: `db.MigrateLogger.Apply(name, stmt)` — success logs Info, "already exists" / "duplicate column name" is silently swallowed, anything else logs Warn but never returns an error (preserving the existing swallow-error contract). + +## Code-quality limits + +Enforced by `apps/backend/.golangci.yml` (errors on new code only): +- Functions: ≤80 lines, ≤50 statements +- Cyclomatic complexity: ≤15 · Cognitive complexity: ≤30 +- Nesting depth: ≤5 · Naked returns only in functions ≤30 lines +- No duplicated blocks (≥150 tokens) · Repeated strings → constants (≥3 occurrences) + +When you hit a limit, extract a helper function. Prefer composition over growing a single function. + +## Further scoped notes + +- `internal/agentctl/AGENTS.md` — agentctl server route groups, adapter model, ACP protocol +- `internal/agentctl/server/api/AGENTS.md` — reverse-proxy body rewriting (`Accept-Encoding`), iframe-blocking header stripping +- `internal/integrations/AGENTS.md` — playbook for adding a new third-party integration (Jira/Linear pattern) diff --git a/apps/backend/CLAUDE.md b/apps/backend/CLAUDE.md new file mode 120000 index 0000000000..47dc3e3d86 --- /dev/null +++ b/apps/backend/CLAUDE.md @@ -0,0 +1 @@ +AGENTS.md \ No newline at end of file diff --git a/apps/backend/go.mod b/apps/backend/go.mod index 03b8eea687..5edb60e8d9 100644 --- a/apps/backend/go.mod +++ b/apps/backend/go.mod @@ -27,6 +27,7 @@ require ( go.opentelemetry.io/otel/sdk v1.43.0 go.opentelemetry.io/otel/trace v1.43.0 go.uber.org/zap v1.26.0 + golang.org/x/net v0.52.0 golang.org/x/sync v0.20.0 golang.org/x/sys v0.42.0 gopkg.in/natefinch/lumberjack.v2 v2.2.1 @@ -102,7 +103,6 @@ require ( go.uber.org/multierr v1.10.0 // indirect golang.org/x/arch v0.3.0 // indirect golang.org/x/crypto v0.49.0 // indirect - golang.org/x/net v0.52.0 // indirect golang.org/x/term v0.41.0 // indirect golang.org/x/text v0.35.0 // indirect google.golang.org/genproto/googleapis/api v0.0.0-20260401024825-9d38bb4040a9 // indirect diff --git a/apps/backend/internal/agentctl/AGENTS.md b/apps/backend/internal/agentctl/AGENTS.md new file mode 100644 index 0000000000..69e1722600 --- /dev/null +++ b/apps/backend/internal/agentctl/AGENTS.md @@ -0,0 +1,32 @@ +# agentctl — HTTP server, adapters, ACP protocol + +Scoped guidance for `apps/backend/internal/agentctl/`. Higher-level backend architecture is in `apps/backend/AGENTS.md`. + +## API Groups + +agentctl exposes these route groups (see `server/api/`): +- `/health`, `/info`, `/status` - Health and status +- `/instances/*` - Multi-instance management +- `/processes/*` - Agent subprocess management (start/stop) +- `/agent/configure`, `/agent/stream` - Agent configuration and event streaming +- `/git/*` - Git operations (status, commit, push, pull, rebase, stage, etc.) +- `/shell/*` - Shell session management +- `/workspace/*` - File operations, search, tree +- `/vscode/*` - VS Code integration proxy + +## Adapter Model + +Protocol adapters in `server/adapter/transport/` normalize different agent CLIs: +- `AgentAdapter` interface defines `Start()`, `Stop()`, `Prompt()`, `Cancel()` +- Transports: `acp` (Claude Code), `codex` (OpenAI Codex), `opencode`, `shared`, `streamjson` +- Top-level adapters: `CopilotAdapter` (GitHub Copilot SDK), `AmpAdapter` (Sourcegraph Amp) +- `process.Manager` owns subprocess, wires stdio to adapter +- Factory pattern in `server/adapter/factory.go` selects adapter by agent type + +## ACP Protocol + +JSON-RPC 2.0 over stdin/stdout between agentctl and agent process. Requests: `initialize`, `session/new`, `session/load`, `session/prompt`, `session/cancel`. Notifications: `session/update` with types `message_chunk`, `tool_call`, `tool_update`, `complete`, `error`, `permission_request`, `context_window`. + +## Further scoped notes + +- `server/api/AGENTS.md` — reverse-proxy body rewriting (`Accept-Encoding`) and iframe-blocking header stripping. diff --git a/apps/backend/internal/agentctl/CLAUDE.md b/apps/backend/internal/agentctl/CLAUDE.md new file mode 120000 index 0000000000..47dc3e3d86 --- /dev/null +++ b/apps/backend/internal/agentctl/CLAUDE.md @@ -0,0 +1 @@ +AGENTS.md \ No newline at end of file diff --git a/apps/backend/internal/agentctl/server/api/AGENTS.md b/apps/backend/internal/agentctl/server/api/AGENTS.md new file mode 100644 index 0000000000..4966ae5967 --- /dev/null +++ b/apps/backend/internal/agentctl/server/api/AGENTS.md @@ -0,0 +1,26 @@ +# agentctl API package notes + +Scoped guidance for `apps/backend/internal/agentctl/server/api/`. Covers patterns that aren't obvious from reading the code in isolation. + +## Reverse-proxy body rewriting (`port_proxy.go`) + +The port proxy's `ModifyResponse` hook scans HTML responses to inject the inspector script (`html_injector.go`). For that scan to work, the body must arrive uncompressed. + +**Don't decompress manually.** Instead, in the `Rewrite` hook, delete `Accept-Encoding` on the outbound request: + +```go +r.Out.Header.Del("Accept-Encoding") +``` + +`net/http.Transport` then adds its own `Accept-Encoding: gzip` and auto-decompresses *before* `ModifyResponse` runs. Leaving the client's value intact (e.g. `gzip, br, deflate`) skips Transport's auto-decompression — the body arrives compressed and any byte scan corrupts it. + +After rewriting the body in `injectScriptsIntoResponse`: + +- `resp.Header.Del("Content-Encoding")` +- Set both `resp.Header["Content-Length"]` and `resp.ContentLength` (forgetting either causes chunked-encoding mismatches in the browser). + +Tested in `port_proxy_test.go::TestCreatePortProxy_StripsAcceptEncodingAndInjectsHTML`. + +## Iframe-blocking headers + +`stripIframeSecurityHeaders` removes `Content-Security-Policy`, `Content-Security-Policy-Report-Only`, and `X-Frame-Options` from proxied HTML responses. This is scoped to HTML responses only (gated by `Content-Type` in `ModifyResponse`); other content types pass through with headers intact. diff --git a/apps/backend/internal/agentctl/server/api/CLAUDE.md b/apps/backend/internal/agentctl/server/api/CLAUDE.md new file mode 120000 index 0000000000..47dc3e3d86 --- /dev/null +++ b/apps/backend/internal/agentctl/server/api/CLAUDE.md @@ -0,0 +1 @@ +AGENTS.md \ No newline at end of file diff --git a/apps/backend/internal/agentctl/server/api/html_injector.go b/apps/backend/internal/agentctl/server/api/html_injector.go new file mode 100644 index 0000000000..dc613ea680 --- /dev/null +++ b/apps/backend/internal/agentctl/server/api/html_injector.go @@ -0,0 +1,56 @@ +package api + +import ( + "bytes" + _ "embed" + "io" + "net/http" + "strconv" + "strings" +) + +//go:embed scripts/inspector.js +var inspectorScript string + +// stripIframeSecurityHeaders removes headers that prevent iframe embedding +// or block injected inline scripts from running. +func stripIframeSecurityHeaders(h http.Header) { + h.Del("Content-Security-Policy") + h.Del("Content-Security-Policy-Report-Only") + h.Del("X-Frame-Options") +} + +// injectInspectorScript inserts the inspector " + pos := strings.LastIndex(strings.ToLower(s), "") + if pos >= 0 { + return []byte(s[:pos] + tag + s[pos:]) + } + return append(html, []byte(tag)...) +} + +// injectScriptsIntoResponse modifies an HTML response in-place: strips +// iframe-blocking headers and injects the inspector script. Assumes the +// caller cleared the outbound Accept-Encoding header so net/http.Transport +// auto-decompresses any upstream gzip before this runs. +func injectScriptsIntoResponse(resp *http.Response) error { + stripIframeSecurityHeaders(resp.Header) + + body, err := io.ReadAll(resp.Body) + if closeErr := resp.Body.Close(); closeErr != nil && err == nil { + err = closeErr + } + if err != nil { + return err + } + + modified := injectInspectorScript(body) + resp.Body = io.NopCloser(bytes.NewReader(modified)) + resp.Header.Del("Content-Encoding") + resp.Header.Set("Content-Length", strconv.Itoa(len(modified))) + resp.ContentLength = int64(len(modified)) + return nil +} diff --git a/apps/backend/internal/agentctl/server/api/html_injector_test.go b/apps/backend/internal/agentctl/server/api/html_injector_test.go new file mode 100644 index 0000000000..5e6ddb788b --- /dev/null +++ b/apps/backend/internal/agentctl/server/api/html_injector_test.go @@ -0,0 +1,103 @@ +package api + +import ( + "io" + "net/http" + "strings" + "testing" +) + +func TestInjectInspectorScript_BeforeBodyClose(t *testing.T) { + html := []byte("

Hello

") + result := injectInspectorScript(html) + s := string(result) + if !strings.Contains(s, "` + +// runtimeShim returns the runtime shim script tag with the given proxy prefix +// baked in. %q produces a JS-safe double-quoted string literal (slashes and +// alphanumerics need no escaping, which matches every prefix we emit). +func runtimeShim(prefix string) string { + return fmt.Sprintf(runtimeShimTemplate, prefix) +} + +// urlInCSSPattern matches `url(...)` invocations in CSS where the argument is +// a path-absolute URL we want to rewrite. The argument can be optionally +// wrapped in single or double quotes and may be surrounded by whitespace. +// +// url(/foo) +// url('/foo') +// url("/foo") +// +// Network-relative (`//host/foo`) and absolute (`http://...`) are left alone. +var urlInCSSPattern = regexp.MustCompile(`url\(\s*(['"]?)(/[^/'"][^'")]*)`) + +// importInCSSPattern matches `@import "/foo";` style root-absolute imports. +var importInCSSPattern = regexp.MustCompile(`@import\s+(['"])(/[^/'"][^'"]*)['"]`) + +// rewriteProxyResponse mutates an `http.Response` from agentctl in place, +// rewriting root-absolute URLs to be prefixed by `proxyPrefix` so the iframe's +// asset/XHR/import requests come back through the same port proxy instead of +// hitting the host page's origin. Returns nil and leaves the response untouched +// for content types we don't rewrite (everything except HTML and CSS today). +// +// `proxyPrefix` is the public URL path that fronts this proxy on the gateway, +// e.g. "/port-proxy//" (no trailing slash). It is prepended to +// matched URLs that start with a single "/" — see `rewriteAbsolutePath`. +func rewriteProxyResponse(resp *http.Response, proxyPrefix string) error { + ct := strings.ToLower(resp.Header.Get("Content-Type")) + var rewrite func([]byte, string) []byte + switch { + case strings.Contains(ct, "text/html"): + rewrite = rewriteHTMLURLs + case strings.Contains(ct, "text/css"): + rewrite = rewriteCSSURLs + default: + return nil + } + + body, err := io.ReadAll(resp.Body) + closeErr := resp.Body.Close() + if err != nil { + return err + } + if closeErr != nil { + return closeErr + } + + modified := rewrite(body, proxyPrefix) + resp.Body = io.NopCloser(bytes.NewReader(modified)) + resp.Header.Del("Content-Encoding") + resp.Header.Set("Content-Length", strconv.Itoa(len(modified))) + resp.ContentLength = int64(len(modified)) + return nil +} + +// rewriteAbsolutePath turns a path-absolute URL ("/foo") into a proxied path +// ("/foo"). Returns the input unchanged for non-rewritable cases: +// empty strings, network-relative URLs (`//host`), schemes (`http:`, `data:`, +// `mailto:`, etc.), or relative paths (`foo`, `./foo`, `../foo`). +func rewriteAbsolutePath(rawURL, prefix string) string { + if len(rawURL) < 1 || rawURL[0] != '/' { + return rawURL + } + if len(rawURL) >= 2 && rawURL[1] == '/' { + return rawURL // network-relative + } + return prefix + rawURL +} + +// htmlRewriteState tracks per-document position during a single +// `rewriteHTMLURLs` pass. Pulled out to its own type so each token-handling +// case stays short and the top-level loop's cyclomatic complexity stays under +// the lint budget. +type htmlRewriteState struct { + out *bytes.Buffer + prefix string + shim string + inStyle bool + inScript bool + shimInjected bool +} + +// onStartTag emits the (URL-rewritten) start tag and updates raw-text +// element tracking. Also injects the runtime shim immediately after ``. +func (s *htmlRewriteState) onStartTag(token html.Token) { + rewriteTokenURLs(&token, s.prefix) + s.out.WriteString(token.String()) + if !s.shimInjected && token.Data == headTagName { + s.out.WriteString(s.shim) + s.shimInjected = true + } + switch token.Data { + case styleTagName: + s.inStyle = true + case scriptTagName: + s.inScript = true + } +} + +// onEndTag clears raw-text element tracking, then writes the end tag. +func (s *htmlRewriteState) onEndTag(token html.Token) { + switch token.Data { + case styleTagName: + s.inStyle = false + case scriptTagName: + s.inScript = false + } + s.out.WriteString(token.String()) +} + +// onTextToken writes text content with the right escaping for the current +// element context: rewrite CSS URLs inside ` + + + +About +External +
+
+ +` + + got := string(rewriteHTMLURLs([]byte(in), proxyPrefix)) + + mustContain(t, got, `href="/port-proxy/abc/3001/styles/main.css"`) + mustContain(t, got, `href="//cdn.example.com/lib.css"`) + mustContain(t, got, `src="/port-proxy/abc/3001/static/app.js"`) + mustContain(t, got, `src="/port-proxy/abc/3001/img/logo.png"`) + mustContain(t, got, `/port-proxy/abc/3001/img/logo@2x.png 2x`) + mustContain(t, got, `https://cdn.example.com/logo.png 3x`) + mustContain(t, got, `href="/port-proxy/abc/3001/about"`) + mustContain(t, got, `href="http://external.example.com/x"`) + mustContain(t, got, `action="/port-proxy/abc/3001/submit"`) + mustContain(t, got, `formaction="/port-proxy/abc/3001/quick"`) + // Inline style="url('/bg.jpg')" — html package HTML-escapes single quotes + // on serialization, so check for the rewritten path without the quote. + mustContain(t, got, `/port-proxy/abc/3001/bg.jpg`) + // Inline