João Salavisa
e145d48f22
* feat: add plugin system with grpc runtime and native ui plugins - go plugin backends ship as release tarballs (per-platform binaries, checksums) that kandev installs from url/upload or filesystem sideload+sync, then spawns and supervises via hashicorp/go-plugin (unix socket / loopback tcp+automtls) - strict typed grpc contract (kandev.plugin.v1): DeliverEvent/InvokeTool/HandleWebhook plus a capability-gated host service (state, secrets, emit); author sdk in pkg/pluginsdk; packaging cli in cmd/plugin-pack - native frontend bundles: window.registerKandevPlugin + registry/host loader with dynamic routes, nav items, slots and ws handlers; plugins settings page with install dialog, sync button and enable/disable/uninstall - gated behind the plugins feature flag (prod off, dev/e2e on); covered by unit and integration tests plus playwright e2e against a real packaged fixture plugin Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(plugins): hydrate base app context on plugin routes Plugin-owned SPA routes classify as RouteUnknown, so the boot payload shipped only feature flags — no workspaces, workflows, or kanban context. Native plugin UI (e.g. host.ui.TaskCreateDialog) then had an empty store. Hydrate the same base context as the Home route for unknown SPA routes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(plugins): expose TaskCreateDialog on host.ui Lets a native plugin open kandev's real create-task modal (prefilled via initialValues) instead of POSTing directly. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(plugins): harden filesystem/URL sinks against CodeQL findings Add sink-level guards on the plugin trust boundary: securejoin containment on archive extraction (zip-slip), plugin-id validation before every fs_store path, http/https scheme validation on install-from-URL, and non-interpolated ws-handler error logging. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(plugins): address review findings (races, resource caps, lifecycle) - serialize per-plugin startup to close a TOCTOU double-spawn race, and let a plugin restart after exhausting attempts - bound plugin startup with an explicit 30s timeout - reject backslash paths and cap decompressed bytes by actual size in pkgtar; only mark a package Signed when a verifier actually ran and passed - delete plugin_state on uninstall; validate webhook keys against the manifest and cap webhook body size; restart an active plugin when a new version installs Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(plugins): sanitize plugin-id path sinks for CodeQL Pin the plugin id to a single path segment with filepath.Base at recordPath/ configPath and pkgtar.Remove — a barrier the taint analysis recognizes. Behaviour-identical for valid ids (already validated by safePluginID); clears the go/path-injection alerts. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(plugins): inline plugin-id validation at path sinks for CodeQL filepath.Base does not strip '..', so it is not a path-injection barrier. Call the recognized safePluginID / id-pattern guard inline in Delete/GetConfig/ SetConfig and pkgtar.Remove — the same barrier that already cleared the Get sink. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(plugins): address code-review findings (backend) - add capability-gating tests for the Host RPC service (was untested) - close a stop-during-start orphan race so Disable/Uninstall can't leave a supervised subprocess behind - on failed upgrade, remove only the new version dir (not all versions + data); restart the prior version - validate manifest version and reject ids ending in .config (record-file alias) - make Deliverer.Refresh non-blocking and cancel in-flight delivery on worker stop - clamp out-of-range webhook status, skip-and-warn corrupt records in List, write records atomically, semver-aware sideload version pick Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(plugins): address code-review findings (frontend) - wrap plugin routes + settings routes in a shared PluginErrorBoundary so a throwing plugin page shows a fallback instead of white-screening the SPA - fix disable/re-enable in one session: keep the cached registration and re-run initialize instead of relying on a cached ES-module re-import - prefix root-relative bundle/style URLs with apiBaseUrl (split-origin/Tauri) - surface partial-install warnings via toast.warning instead of a bare success - dedupe/remove injected style links on unload; gate PluginNavItems on the flag Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(plugins): serialize per-plugin lifecycle + guard config writes - add a per-plugin keyed lock in Service so concurrent Enable/Disable/Install/ Uninstall/UpdateConfig for one id are serialized (no status-machine race) - take s.mu in SetConfig/GetConfig and write config atomically - bound each plugin initialize() with a timeout so one hung bundle can't stall the rest of the boot load Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(plugins): split service_test.go to satisfy file-length lint Move the install/URL test group into service_install_test.go; no logic change. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * ci: build and ship the e2e fixture plugin package to E2E shards The plugin E2E specs' global-setup requires apps/backend/.build/kandev-plugin-e2e-1.0.0.tar.gz. Build it in the E2E build job, upload it as an artifact, and download it in both the regular and container shard jobs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Kandev Agent <agent@kandev.dev> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
178 lines
10 KiB
Markdown
178 lines
10 KiB
Markdown
# Plan: Office Plugin System (Phase 1 + UI iframe proxy)
|
||
|
||
**Spec:** `docs/specs/plugins/spec.md`
|
||
**Status:** in progress
|
||
**Owner:** jcfs
|
||
|
||
## Goal
|
||
|
||
Implement the plugin system end-to-end for an operator to register an
|
||
out-of-process HTTP plugin, have kandev deliver signed events to it, invoke its
|
||
tools, proxy external webhooks and iframe UI pages to it, expose plugin-scoped KV
|
||
state + secrets, monitor health, and manage it from a Plugins settings page —
|
||
all gated behind a new `plugins` feature flag.
|
||
|
||
## In scope
|
||
|
||
- Backend `internal/plugins/` package (provider pattern): manifest types +
|
||
validation, filesystem registration store, in-memory registry, lifecycle state
|
||
machine, health poller, event delivery (HMAC + retries + ring buffer), tool
|
||
invocation client, webhook + UI reverse proxies, `plugin_state` SQLite store,
|
||
auth middleware + capability checks, secrets resolution.
|
||
- `/api/plugins/*` management + plugin-facing API.
|
||
- `plugins` feature flag (profiles.yaml, `FeaturesConfig`, runtimeflags registry,
|
||
frontend `FeatureFlags`).
|
||
- Frontend Plugins settings page + plugin detail with sandboxed iframe pages.
|
||
- Fixture plugin binary for tests/e2e.
|
||
- Playwright e2e.
|
||
|
||
## SCOPE REVISION — option C (native JS UI plugins)
|
||
|
||
User chose the full Mattermost-webapp model: plugins ship JS bundles loaded into
|
||
the running SPA that register **native** routes/nav/components/reducers (goal: a
|
||
plugin can implement `/jira` as a native page). Iframe pages are dropped in favor
|
||
of this. The frozen frontend contract is `docs/plans/plugins/PLUGIN-API.md`.
|
||
Backend foundations (Wave 1 + task-05 service) are unchanged and reused. Waves
|
||
below are superseded by the "Option C waves" section.
|
||
|
||
Still out of scope: marketplace/index, managed process runtime (kandev spawning
|
||
plugins), wiring plugin tools into live agent sessions (client + listing only),
|
||
rate limiting, plugin DB namespaces, hot reload, multi-instance, and plugin-JS
|
||
sandboxing (workers/realms — documented as future work; v1 loads only active
|
||
operator-registered plugins over same-origin proxy).
|
||
|
||
## Architecture anchors (verified)
|
||
|
||
- HTTP: gin; `registerRoutes(routeParams)` in
|
||
`apps/backend/internal/backendapp/helpers.go:478`; domain groups follow
|
||
`jira.RegisterRoutes(router, dispatcher, svc, log)` → `router.Group("/api/v1/jira")`.
|
||
- Service wiring: `apps/backend/internal/backendapp/services.go` `provideServices(...)`
|
||
with `init<Name>Service(...)` helpers calling per-package `Provide(...)`; result
|
||
fields land on `Services` and `routeParams`.
|
||
- Event bus: `internal/events/bus.EventBus` (`Subscribe(subject, handler) (Subscription, error)`,
|
||
wildcard subjects supported); model delivery after
|
||
`internal/gateway/websocket/task_notifications.go` `RegisterTaskNotifications`.
|
||
Event names in `internal/events/types.go` (`events.TaskCreated="task.created"`, etc).
|
||
- Health: `internal/integrations/healthpoll` `Prober`/`Poller` (interval configurable).
|
||
- Secrets: `internal/secrets` + `internal/integrations/secretadapter.New(store)`.
|
||
- Persistence: SQLite; per-package `initSchema()` (see
|
||
`internal/task/repository/sqlite/base_schema.go`). No central migrations.
|
||
- UI proxy precedent: `internal/agentctl/server/api/` reverse proxy (header stripping).
|
||
- Feature flag pattern: `profiles.yaml` `features:` block →
|
||
`internal/common/config/config.go` `FeaturesConfig` (`config.go:174`) →
|
||
`internal/runtimeflags/registry.go` + `runtimeflags/config.go:31` →
|
||
frontend `apps/web/lib/state/slices/features/types.ts` + `hooks/domains/features/use-feature.ts`.
|
||
|
||
## Waves
|
||
|
||
### Wave 1 — backend foundations + fixture (parallel; disjoint files)
|
||
- **task-01** manifest types + validation (`internal/plugins/manifest/`)
|
||
- **task-02** filesystem registration store + credentials (`internal/plugins/store/`)
|
||
- **task-03** `plugin_state` SQLite store (`internal/plugins/state/`)
|
||
- **task-04** fixture plugin binary (`apps/backend/cmd/plugin-fixture/`)
|
||
|
||
### Wave 2 — service, delivery, API, proxies (sequential within package)
|
||
- **task-05** registry + service + lifecycle state machine + `Provide` (`internal/plugins/service.go`, `registry.go`, `provider.go`)
|
||
- **task-06** event delivery subscriber (HMAC, retries, ring buffer) (`internal/plugins/delivery/`)
|
||
- **task-07** health poller integration (`internal/plugins/health.go`)
|
||
- **task-08** tool invocation client + webhook & UI reverse proxies (`internal/plugins/client.go`, `proxy.go`)
|
||
- **task-09** HTTP API + auth middleware + capability checks + backendapp wiring (`internal/plugins/handlers.go`, `middleware.go`; edits to `backendapp/services.go`, `helpers.go`)
|
||
|
||
### Wave 3 — feature flag + frontend
|
||
- **task-10** `plugins` feature flag (backend + frontend plumbing)
|
||
- **task-11** plugins API client + Zustand slice (+ unit tests)
|
||
- **task-12** Plugins settings page (list/register/enable/disable/uninstall) + route
|
||
- **task-13** plugin detail with sandboxed iframe pages + postMessage bridge
|
||
|
||
### Wave 4 — e2e + verify
|
||
- **task-14** Playwright e2e (register→active, enable/disable, iframe renders, event delivery)
|
||
- **task-15** spec update + docs + full `make fmt typecheck test lint` + web typecheck/lint
|
||
|
||
## Option C waves (ACTIVE)
|
||
|
||
Contract: `PLUGIN-API.md`. task-12/13 (iframe) superseded by task-19/20.
|
||
|
||
### Wave A — foundations (parallel; disjoint)
|
||
- **task-06** event delivery (unchanged) — `internal/plugins/delivery/`
|
||
- **task-16** manifest `ui.bundle`/`ui.styles` fields — `internal/plugins/manifest/`
|
||
- **task-10** `plugins` feature flag (backend+frontend plumbing)
|
||
- **task-18** frontend Plugin API: globals + registry + host/loader — `apps/web/lib/plugins/`
|
||
|
||
### Wave B — integration (after A)
|
||
- **task-17** backend API + auth/caps + state/secrets + webhook proxy + bundle proxy +
|
||
boot-payload `plugins` + health + backendapp wiring (merges old 07/08/09) —
|
||
needs task-06, task-16, task-10
|
||
- **task-19** frontend dynamic integration: routes/nav/slots/ws bridge + PluginSlot —
|
||
needs task-18
|
||
- **task-20** plugins management page (list/register/enable/disable) + API client +
|
||
slice — needs task-18, task-17
|
||
|
||
### Wave C — example + e2e + verify (after B)
|
||
- **task-21** example plugin repo (git init; backend + JS bundle + manifest + README) —
|
||
needs PLUGIN-API.md, task-16
|
||
- **task-22** Playwright e2e loading the example plugin: native nav item + page +
|
||
slot + WS-driven update — needs task-17, task-19, task-20, task-21
|
||
- **task-23** spec + docs update + full verify — needs all
|
||
|
||
## gRPC transport waves (ACTIVE)
|
||
|
||
Contract: `GRPC-CONTRACT.md` (frozen). Supersedes the HTTP+HMAC backend transport
|
||
from Wave 2 / Option C Wave B's task-17: the plugin **backend** now speaks gRPC over
|
||
a kandev-spawned `hashicorp/go-plugin` subprocess instead of HTTP with
|
||
API-key/HMAC-signed webhooks. Registration-by-manifest-paste
|
||
(`POST /api/plugins/register`, generated `api_key`/`webhook_secret`) is removed
|
||
outright — replaced by install-by-URL/upload of a release tarball. task-06
|
||
(event delivery semantics), task-16 (manifest `ui.bundle`/`ui.styles`), task-10
|
||
(feature flag), and all of Wave A's frontend plugin-host work (task-18/19/20's UI
|
||
layer) carry over unchanged; only the backend transport and the install flow change.
|
||
|
||
- **G1 — proto + SDK.** Author `apps/backend/proto/kandev/plugin/v1/plugin.proto`
|
||
(`Plugin`/`Host` services, all messages per GRPC-CONTRACT.md §3), generate Go
|
||
stubs, and build `apps/backend/pkg/pluginsdk` (§4: `Plugin`/`Host` interfaces,
|
||
`Serve(p Plugin, opts...)`, `UnimplementedPlugin`). No transport wiring yet.
|
||
- **G2 — installer package.** Package format + install pipeline (§6):
|
||
checksum verification, optional ed25519 signature check (unsigned → warn),
|
||
manifest validation before any code runs, extraction to
|
||
`~/.kandev/plugins/<id>/<version>/`, and the on-disk install record
|
||
(id/version/install_path/status). No spawning yet.
|
||
- **G3 — runtime manager + host service + transport swap + install API + wiring.**
|
||
go-plugin process manager (spawn, handshake §2, `Ping`-based health, crash/backoff
|
||
restart with the state machine in the spec), the `Host` gRPC service
|
||
implementation (GetState/SetState/DeleteState/ListState/RevealSecret/EmitEvent)
|
||
with the capability-gating server interceptor, swap event delivery / tool
|
||
invocation / webhook proxy from HTTP calls to `DeliverEvent`/`InvokeTool`/
|
||
`HandleWebhook` gRPC calls, `POST /api/plugins/install` handler (JSON `{url}` or
|
||
multipart `package`), removal of `POST /api/plugins/register` and all
|
||
api_key/webhook_secret/HMAC code paths, and `backendapp` wiring.
|
||
- **G4 — fixture + hello rewrite + frontend install UI.** Rewrite
|
||
`apps/backend/cmd/plugin-fixture/` as a `pluginsdk`-based Go binary (drop the
|
||
HTTP fixture server); rewrite the `kandev-plugin-hello` example repo's backend
|
||
half the same way (frontend bundle unchanged per PLUGIN-API.md §7); replace the
|
||
frontend "Register plugin" flow (manifest paste, credential display) with an
|
||
"Install plugin" flow (URL input + file upload, no credentials shown) per
|
||
PLUGIN-API.md's updated "Loading model"/"Security posture".
|
||
- **G5 — e2e + verify.** Playwright coverage for install-from-url, install-by-upload,
|
||
crash → restart → recovery (buffered event flush), and capability-denied
|
||
(gRPC `PermissionDenied`) scenarios; full `make fmt typecheck test lint` plus web
|
||
typecheck/lint/test.
|
||
|
||
## Verification
|
||
|
||
- Backend: `make -C apps/backend test`, `make -C apps/backend lint`, `make -C apps/backend build`
|
||
- Web: `cd apps/web && pnpm run typecheck && pnpm lint && pnpm test`
|
||
- E2E: `cd apps/web && pnpm e2e -g "plugin"`
|
||
- Final: `make -C apps/backend fmt` then `make -C apps/backend typecheck test lint`
|
||
|
||
## Risks / open questions
|
||
|
||
- Event `payload` shape: bus events carry `event.Data` as struct or
|
||
`map[string]interface{}`. Delivery must normalize to the spec's JSON envelope;
|
||
task-06 defines a `toEnvelope` that marshals `event.Data` under `payload`.
|
||
- `workspace_id` is not present on every event; envelope sets it when derivable,
|
||
else omits. Acceptable for Phase 1.
|
||
- Sequential-per-plugin delivery must not block the bus goroutine; task-06 uses a
|
||
per-plugin queue + worker.
|
||
- Feature-flag default must keep prod OFF; e2e/dev ON so tests run.
|
||
|
||
**Status 2026-07-16:** all gRPC waves (G1–G5) completed. E2E green (2 specs, 3x repeat no flake); full backend+web gate green. One integration bug found by e2e and fixed (install response unwrapping in plugins-api.ts).
|