Files
kandev/apps/backend/internal/plugins/store/fs_store.go
T
João Salavisa e145d48f22 feat: add plugin system with grpc runtime and native ui plugins (#1742)
* feat: add plugin system with grpc runtime and native ui plugins

- go plugin backends ship as release tarballs (per-platform binaries, checksums) that
  kandev installs from url/upload or filesystem sideload+sync, then spawns and
  supervises via hashicorp/go-plugin (unix socket / loopback tcp+automtls)
- strict typed grpc contract (kandev.plugin.v1): DeliverEvent/InvokeTool/HandleWebhook
  plus a capability-gated host service (state, secrets, emit); author sdk in
  pkg/pluginsdk; packaging cli in cmd/plugin-pack
- native frontend bundles: window.registerKandevPlugin + registry/host loader with
  dynamic routes, nav items, slots and ws handlers; plugins settings page with
  install dialog, sync button and enable/disable/uninstall
- gated behind the plugins feature flag (prod off, dev/e2e on); covered by unit and
  integration tests plus playwright e2e against a real packaged fixture plugin

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(plugins): hydrate base app context on plugin routes

Plugin-owned SPA routes classify as RouteUnknown, so the boot payload shipped
only feature flags — no workspaces, workflows, or kanban context. Native plugin
UI (e.g. host.ui.TaskCreateDialog) then had an empty store. Hydrate the same
base context as the Home route for unknown SPA routes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(plugins): expose TaskCreateDialog on host.ui

Lets a native plugin open kandev's real create-task modal (prefilled via
initialValues) instead of POSTing directly.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(plugins): harden filesystem/URL sinks against CodeQL findings

Add sink-level guards on the plugin trust boundary: securejoin containment on
archive extraction (zip-slip), plugin-id validation before every fs_store path,
http/https scheme validation on install-from-URL, and non-interpolated ws-handler
error logging.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(plugins): address review findings (races, resource caps, lifecycle)

- serialize per-plugin startup to close a TOCTOU double-spawn race, and let a
  plugin restart after exhausting attempts
- bound plugin startup with an explicit 30s timeout
- reject backslash paths and cap decompressed bytes by actual size in pkgtar;
  only mark a package Signed when a verifier actually ran and passed
- delete plugin_state on uninstall; validate webhook keys against the manifest
  and cap webhook body size; restart an active plugin when a new version installs

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(plugins): sanitize plugin-id path sinks for CodeQL

Pin the plugin id to a single path segment with filepath.Base at recordPath/
configPath and pkgtar.Remove — a barrier the taint analysis recognizes.
Behaviour-identical for valid ids (already validated by safePluginID); clears
the go/path-injection alerts.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(plugins): inline plugin-id validation at path sinks for CodeQL

filepath.Base does not strip '..', so it is not a path-injection barrier. Call
the recognized safePluginID / id-pattern guard inline in Delete/GetConfig/
SetConfig and pkgtar.Remove — the same barrier that already cleared the Get sink.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(plugins): address code-review findings (backend)

- add capability-gating tests for the Host RPC service (was untested)
- close a stop-during-start orphan race so Disable/Uninstall can't leave a
  supervised subprocess behind
- on failed upgrade, remove only the new version dir (not all versions + data);
  restart the prior version
- validate manifest version and reject ids ending in .config (record-file alias)
- make Deliverer.Refresh non-blocking and cancel in-flight delivery on worker stop
- clamp out-of-range webhook status, skip-and-warn corrupt records in List,
  write records atomically, semver-aware sideload version pick

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(plugins): address code-review findings (frontend)

- wrap plugin routes + settings routes in a shared PluginErrorBoundary so a
  throwing plugin page shows a fallback instead of white-screening the SPA
- fix disable/re-enable in one session: keep the cached registration and
  re-run initialize instead of relying on a cached ES-module re-import
- prefix root-relative bundle/style URLs with apiBaseUrl (split-origin/Tauri)
- surface partial-install warnings via toast.warning instead of a bare success
- dedupe/remove injected style links on unload; gate PluginNavItems on the flag

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(plugins): serialize per-plugin lifecycle + guard config writes

- add a per-plugin keyed lock in Service so concurrent Enable/Disable/Install/
  Uninstall/UpdateConfig for one id are serialized (no status-machine race)
- take s.mu in SetConfig/GetConfig and write config atomically
- bound each plugin initialize() with a timeout so one hung bundle can't stall
  the rest of the boot load

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* test(plugins): split service_test.go to satisfy file-length lint

Move the install/URL test group into service_install_test.go; no logic change.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* ci: build and ship the e2e fixture plugin package to E2E shards

The plugin E2E specs' global-setup requires apps/backend/.build/kandev-plugin-e2e-1.0.0.tar.gz.
Build it in the E2E build job, upload it as an artifact, and download it in both
the regular and container shard jobs.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Kandev Agent <agent@kandev.dev>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 12:00:26 +01:00

295 lines
9.1 KiB
Go

package store
import (
"fmt"
"os"
"path/filepath"
"regexp"
"strings"
"sync"
"go.uber.org/zap"
"gopkg.in/yaml.v3"
"github.com/kandev/kandev/internal/common/logger"
)
// idPattern mirrors internal/plugins/manifest's plugin id pattern:
// lowercase alphanumerics, dots, underscores, and hyphens, starting with a
// lowercase alphanumeric. manifest.Validate already enforces this shape
// upstream during install; this is the sink-level guard every FSStore
// method re-applies to id before building a filesystem path from it, so a
// malformed id (e.g. containing "..", "/", or "\") can never reach disk
// I/O regardless of how it got here.
var idPattern = regexp.MustCompile(`^[a-z0-9][a-z0-9._-]*$`)
// dotConfigSuffix is the suffix a plugin id must never end with: id+".yml"
// would then collide with the "<id>.config.yml" naming convention
// isRecordFile uses to distinguish a plugin's own record from another
// plugin's operator config, silently hiding the record from List() (see
// isRecordFile).
const dotConfigSuffix = ".config"
// safePluginID rejects any id that is not a single, clean path segment
// matching idPattern, or that ends in ".config" (see dotConfigSuffix). It is
// the guard called by every FSStore method before recordPath/configPath
// build a path from id.
func safePluginID(id string) error {
if !idPattern.MatchString(id) {
return fmt.Errorf("plugins: invalid plugin id %q", id)
}
if strings.HasSuffix(id, dotConfigSuffix) {
return fmt.Errorf("plugins: invalid plugin id %q: must not end in %q", id, dotConfigSuffix)
}
return nil
}
// FSStore persists plugin installation records under dir as "{id}.yml" (the
// Record) and "{id}.config.yml" (operator-editable config). FSStore
// implements Store.
type FSStore struct {
dir string
log *logger.Logger
// mu guards every disk write (Save, Delete, SetConfig) and read
// (GetConfig) that touches a plugin's record/config files, so
// concurrent callers (e.g. two operator UpdateConfig PATCH requests for
// the same id) can never interleave writes to the same path. Writers
// take Lock; GetConfig takes RLock since it only reads.
mu sync.RWMutex
}
// NewFSStore returns a store that persists records under dir.
func NewFSStore(dir string) *FSStore {
return &FSStore{dir: dir}
}
// SetLogger wires a logger List uses to warn about a corrupt/unreadable
// record file it has to skip. Optional: a nil (default, unset) logger makes
// List silently skip instead of warning.
func (s *FSStore) SetLogger(log *logger.Logger) {
s.log = log
}
// recordPath returns the on-disk path for a plugin's installation record.
// Every public method validates id with safePluginID before reaching here.
func (s *FSStore) recordPath(id string) string {
return filepath.Join(s.dir, id+".yml")
}
// configPath returns the on-disk path for a plugin's operator config.
func (s *FSStore) configPath(id string) string {
return filepath.Join(s.dir, id+".config.yml")
}
// writeRecord marshals record to YAML and writes it to disk, creating the
// store directory if needed. Used both to create a fresh record (Install)
// and to persist updates to an existing one (Save). The write is atomic
// (tmp file + rename within the same directory) so a crash mid-write can
// never leave a half-written "<id>.yml" for List()/Get() to trip over.
func (s *FSStore) writeRecord(record *Record) error {
if err := safePluginID(record.ID); err != nil {
return err
}
if err := os.MkdirAll(s.dir, 0o755); err != nil {
return fmt.Errorf("create plugin store dir: %w", err)
}
data, err := yaml.Marshal(record)
if err != nil {
return fmt.Errorf("marshal plugin record: %w", err)
}
if err := writeFileAtomic(s.dir, s.recordPath(record.ID), data, 0o600); err != nil {
return fmt.Errorf("write plugin record: %w", err)
}
return nil
}
// writeFileAtomic writes data to path via a temp file created in dir (the
// same filesystem/directory as path, so the final os.Rename is atomic) and
// an fsync-then-rename, so a reader (List/Get) never observes a
// partially-written file. The temp file is removed on any failure before
// the rename.
func writeFileAtomic(dir, path string, data []byte, mode os.FileMode) error {
tmp, err := os.CreateTemp(dir, ".tmp-*")
if err != nil {
return fmt.Errorf("create temp file: %w", err)
}
tmpPath := tmp.Name()
succeeded := false
defer func() {
if !succeeded {
_ = os.Remove(tmpPath)
}
}()
if _, err := tmp.Write(data); err != nil {
_ = tmp.Close()
return fmt.Errorf("write temp file: %w", err)
}
if err := tmp.Sync(); err != nil {
_ = tmp.Close()
return fmt.Errorf("sync temp file: %w", err)
}
if err := tmp.Close(); err != nil {
return fmt.Errorf("close temp file: %w", err)
}
if err := os.Chmod(tmpPath, mode); err != nil {
return fmt.Errorf("chmod temp file: %w", err)
}
if err := os.Rename(tmpPath, path); err != nil {
return fmt.Errorf("rename temp file: %w", err)
}
succeeded = true
return nil
}
// Get loads the record for id from disk.
func (s *FSStore) Get(id string) (*Record, error) {
if err := safePluginID(id); err != nil {
return nil, err
}
data, err := os.ReadFile(s.recordPath(id))
if err != nil {
if os.IsNotExist(err) {
return nil, ErrNotFound
}
return nil, fmt.Errorf("read plugin record: %w", err)
}
var record Record
if err := yaml.Unmarshal(data, &record); err != nil {
return nil, fmt.Errorf("unmarshal plugin record: %w", err)
}
return &record, nil
}
// List returns every installed plugin's record. A record file that fails to
// read or parse (corrupt, stray, or written by a future incompatible
// version) is skipped and warned about rather than failing List
// wholesale — the whole plugin subsystem's boot path depends on List()
// succeeding (Registry.Load), so one bad file must not disable every other
// installed plugin.
func (s *FSStore) List() ([]*Record, error) {
entries, err := os.ReadDir(s.dir)
if err != nil {
if os.IsNotExist(err) {
return nil, nil
}
return nil, fmt.Errorf("read plugin store dir: %w", err)
}
var records []*Record
for _, entry := range entries {
if entry.IsDir() || !isRecordFile(entry.Name()) {
continue
}
id := strings.TrimSuffix(entry.Name(), ".yml")
record, err := s.Get(id)
if err != nil {
s.warnSkippedRecord(id, err)
continue
}
records = append(records, record)
}
return records, nil
}
// warnSkippedRecord logs (if a logger is wired via SetLogger) that List
// skipped id's record file rather than failing outright.
func (s *FSStore) warnSkippedRecord(id string, err error) {
if s.log == nil {
return
}
s.log.Warn("plugins: skipping unreadable plugin record",
zap.String("plugin_id", id), zap.Error(err))
}
// isRecordFile reports whether name is a plugin record file ("{id}.yml"),
// excluding config files ("{id}.config.yml").
func isRecordFile(name string) bool {
return strings.HasSuffix(name, ".yml") && !strings.HasSuffix(name, ".config.yml")
}
// Save writes record to disk, creating it if it does not already exist or
// overwriting it (in full) if it does. Used both to persist a fresh install
// and to persist status/metadata updates to an existing record.
func (s *FSStore) Save(record *Record) error {
s.mu.Lock()
defer s.mu.Unlock()
return s.writeRecord(record)
}
// Delete removes the record and its operator config for id.
func (s *FSStore) Delete(id string) error {
if err := safePluginID(id); err != nil {
return err
}
s.mu.Lock()
defer s.mu.Unlock()
if _, err := s.Get(id); err != nil {
return err
}
if err := os.Remove(s.recordPath(id)); err != nil {
return fmt.Errorf("delete plugin record: %w", err)
}
if err := os.Remove(s.configPath(id)); err != nil && !os.IsNotExist(err) {
return fmt.Errorf("delete plugin config: %w", err)
}
return nil
}
// GetConfig returns the operator-editable config for id, or an empty map if
// no config has been set yet.
func (s *FSStore) GetConfig(id string) (map[string]any, error) {
if err := safePluginID(id); err != nil {
return nil, err
}
s.mu.RLock()
defer s.mu.RUnlock()
if _, err := s.Get(id); err != nil {
return nil, err
}
data, err := os.ReadFile(s.configPath(id))
if err != nil {
if os.IsNotExist(err) {
return map[string]any{}, nil
}
return nil, fmt.Errorf("read plugin config: %w", err)
}
var cfg map[string]any
if err := yaml.Unmarshal(data, &cfg); err != nil {
return nil, fmt.Errorf("unmarshal plugin config: %w", err)
}
if cfg == nil {
cfg = map[string]any{}
}
return cfg, nil
}
// SetConfig replaces the operator-editable config for id. The write is
// atomic (tmp file + rename within the store dir, via writeFileAtomic), the
// same guarantee writeRecord gives Save, and is additionally serialized by
// s.mu so two concurrent SetConfig calls for the same id (e.g. two operator
// UpdateConfig PATCH requests racing) can never interleave.
func (s *FSStore) SetConfig(id string, config map[string]any) error {
if err := safePluginID(id); err != nil {
return err
}
s.mu.Lock()
defer s.mu.Unlock()
if _, err := s.Get(id); err != nil {
return err
}
data, err := yaml.Marshal(config)
if err != nil {
return fmt.Errorf("marshal plugin config: %w", err)
}
if err := writeFileAtomic(s.dir, s.configPath(id), data, 0o644); err != nil {
return fmt.Errorf("write plugin config: %w", err)
}
return nil
}