Files
kandev/apps/backend/internal/agent/runtime/lifecycle/manager_execution.go
T
João Salavisa 919a39e49a feat(auth): opt-in authentication & multi-user segregation (#1930)
* feat(auth): add auth core — identities, sessions, PATs, invites, mode state machine

Phase 1 of opt-in authentication + multi-user segregation:
- internal/auth: argon2id password hashing (PHC), opaque session/PAT tokens
  (SHA-256 at rest), login rate limiter, auth-mode state machine
  (disabled/setup/enabled), setup-wizard promotion of the pre-auth
  default-user row into the admin account, invites, admin user management
  with last-admin guard.
- internal/auth/store: auth_identities (OIDC-ready), auth_sessions,
  auth_api_tokens, auth_invites tables (SQLite+Postgres dialects).
- internal/auth/authn: request-identity plumbing (gin + plain contexts),
  RequireAdmin middleware.
- users table: display_name/role/status columns via new MigrateLogger
  wiring; AccountRepository surface (consumer-scoped, keeps existing
  Repository fakes untouched).
- config: AuthConfig.Required/SessionTTLHours/CookieName
  (KANDEV_AUTH_REQUIRED), FeaturesConfig.Auth; profiles.yaml + runtime
  flag registry entry for KANDEV_FEATURES_AUTH.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(auth): enforce opt-in auth over HTTP — middleware, auth API, boot payload

Phase 2 of opt-in authentication + multi-user segregation:
- internal/auth/httpmw: global gin middleware after CORS. Disabled mode
  injects the synthetic single-user admin identity (behavior unchanged);
  enabled mode resolves session cookie then PAT bearer, with an explicit
  allowlist (health, SPA bootstrap reads, credential endpoints,
  self-authenticating webhooks), deferral for WS/proxy upgrades and
  office agent JWTs, and SPA shell availability for the login page.
- internal/auth/httpapi: full auth surface — setup wizard, login/logout,
  me, password change, sessions, PATs, invites (mint/preview/accept),
  auth settings (enable/disable), admin user management.
- Boot payload: initialState.auth block always present; unauthenticated
  visitors on auth-enabled instances get features+auth only (no data).
- External MCP: PAT required once auth is enabled (open while disabled).
- Admin guards on destructive system routes and runtime-flag overrides.
- Startup warning when bound non-loopback with auth disabled
  (ServerConfig.NonLoopbackBinds fail-closed nudge).
- Wiring: auth store in Repositories, auth service in Services,
  ClaimUnownedWorkspaces backfill on the task repository.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(auth): per-user workspace scoping across HTTP services

Phase 3 of opt-in authentication + multi-user segregation:
- Task service: workspaces, workflows, repositories, and tasks are scoped
  to the request identity (owner-only visibility; pre-auth unowned rows
  stay shared). Denials use *NotFound sentinels - no existence leak.
  Internal callers and the disabled-mode synthetic identity remain
  unscoped, preserving behavior exactly when auth is off.
- CreateWorkspace stamps the authenticated caller as owner.
- Session chokepoint: lifecycle Manager.GetOrEnsureExecution runs a
  settable session-access check covering shell/files/ports/vscode/LSP.
- Secrets: per-user user_id column, scoped CRUD/list, ClaimUnowned wired
  into the setup-wizard backfills.
- Boot payload inherits scoping automatically.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(auth): scope WebSocket gateway to authenticated users

Phase 4 of opt-in authentication + multi-user segregation:
- Connection auth: /ws, /terminal, /lsp, /vscode, /port-proxy routes are
  guarded when auth is enabled - session cookie (same-origin upgrade),
  resolved identity from the HTTP middleware, or ?token=<PAT> for
  headerless programmatic clients; JSON 401 before any upgrade.
- Client identity: every WS client carries its authn.Identity; dispatched
  RPC actions (workspace.list, task CRUD, ...) run under a context
  carrying it, so WS RPC gets the same per-user scoping as HTTP.
- Subscription checks: task/session subscribe+focus verify workspace
  visibility via the task service; user topic subscriptions are pinned to
  the client's own user.
- Broadcast routing: new Hub.BroadcastToWorkspace routes workspace-scoped
  events (task/workflow/repo/office/session-state) to the owner's clients
  only, with safe fallbacks (unowned/unresolvable -> broadcast) so events
  never vanish; instance-wide events (executors, environments, agent
  profiles, system jobs) stay global by design.
- Replaced the long-standing discarded-token TODO in HandleConnection.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(auth): add feature spec and backend architecture notes for opt-in auth

- docs/specs/auth/spec.md: product behavior (opt-in, setup wizard,
  per-user workspaces, sessions/PATs, invites, public surface, v1 limits)
- apps/backend/AGENTS.md: internal/auth package layout + scoping rules
  for new service entry points and WS broadcast call sites

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(auth): public documentation for opt-in authentication

- New docs/public/authentication.md: enabling, users & invites, PATs,
  public endpoints, limitations, disabling
- run-as-a-service.md: network-security note now points at the real
  authentication boundary instead of reverse-proxy-only guidance
- meta.json: nav entry under Configure Kandev

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(auth): ADR for opt-in authentication architecture

Records the load-bearing choices: mode state machine + synthetic identity
in disabled mode, opaque DB-backed credentials over JWTs, default-user
promotion at setup, service-layer scoping with no-admin-bypass, and the
explicit public allowlist.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(web): auth UI — login, setup wizard, invites, users & account settings

Adds the frontend half of opt-in authentication: a pre-auth gate in
main.tsx that renders the setup wizard or login/invite pages instead of
the app shell whenever the boot payload's auth block says so, a new
`auth` store slice + feature flag mirroring the backend's StatePayload,
credentialed fetch (`credentials: "include"`) with a 401 handler that
clears the session and redirects to /login, and a full auth-api.ts
client for the /api/v1/auth and /api/v1/users routes.

Settings additions: a System > Users page (list/create/disable/promote
users, invite-link dialog) and System > Authentication page (enable/
disable with confirm + env-locked banner), gated on the `auth` feature
flag and admin role. A new Account sidebar group (Profile & Password,
API Tokens) is shown to signed-in users once auth is enabled.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(auth): E2E lifecycle suite + fix workspace event WS leak it caught

New dedicated Playwright auth project (backend restarts with
KANDEV_AUTH_REQUIRED=true, isolated from the tokenless default suite):
setup-mode boot gating, admin setup wizard, login/logout + bad
credentials, PATs, admin-surface authorization, anonymous WS rejection,
UI login flow, and the acceptance test - two-user segregation across
HTTP, boot payload, and WebSocket.

The segregation spec immediately caught a real leak: workspace.* event
payloads carry their ID as 'id' (the payload IS the workspace DTO), not
'workspace_id', so BroadcastToWorkspace fell back to broadcast-to-all
for workspace create/update/delete. Fixed with action-aware extraction
plus a unit regression test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(auth): close cross-user access gaps found in security audit

Two independent security audits of the auth branch found several
authorization gaps where session/workspace-scoped surfaces were not wired
to the new per-user chokepoint. All fixed:

- IDOR on reverse-proxy/terminal routes (vscode, port-proxy, terminal):
  these resolve executions via GetExecutionBySessionID /
  GetOrEnsureExecutionForEnvironment / EnsurePassthroughExecution, which
  skipped the session-access check that only GetOrEnsureExecution ran. Added
  the check inside the env/passthrough resolvers (before the cache
  short-circuit) and an explicit CheckSessionAccess gate in the two bare-lookup
  proxy handlers. An authenticated non-owner can no longer reach another
  user's IDE, forwarded ports, or shell.
- Cross-user agent-conversation read: ListMessages / ListMessagesPaginated /
  SearchMessages now authorize the session.
- Cross-user repository-script read/inject: script CRUD now authorizes the
  parent repository's workspace.
- Office HTTP surface: a group middleware enforces workspace ownership on
  :wsId routes for real users (agent-JWT callers keep their claim scoping).
- Login rate-limit bypass: SetTrustedProxies(nil) so a spoofed
  X-Forwarded-For can't rotate the limiter key on a directly-reachable
  backend.
- Invite double-provisioning race: MarkInviteUsed is now the atomic
  serialization point (consumed before user creation).
- Setup wizard: serialized with a mutex so concurrent first-visitors can't
  corrupt the promoted admin profile.
- Log sanitizer: explicit kandev_pat_ redaction rule.

Regression tests added for message/script scoping and the
execution-access-before-cache invariant. Auth E2E (incl. two-user
segregation) still green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(auth): scope repository active-session-count endpoint

GET /api/v1/repositories/:id/active-session-count went through
CountActiveSessionsByRepository, which was not identity-scoped - a
cross-user info leak of another owner's active session counts. Added the
same authorizeRepositoryID gate the script routes use.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(auth): add json tags to auth store models — omit digests, fix token UI

The auth store structs (APIToken, Invite, LoginIdentity, Session) had db:
tags but no json: tags, so the handlers that serialize them directly sent
Go-cased keys (CreatedAt, Name) AND leaked the SHA-256/password digests to
clients. Two consequences:
- The API-tokens UI showed 'Invalid Date' and an empty name (frontend reads
  created_at/name).
- token_sha256 / password_hash were serialized in list responses.

Added json tags with snake_case keys the frontend reads and json:"-" on
every credential digest. Regression test asserts digests never serialize and
the snake_case timestamp/name keys are present. Also adds the
auth-screenshots capture spec (artifacts gitignored).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(web): move auth enable into feature toggles; show current user in sidebar

Remove the standalone Authentication settings page and its dedicated
settings/get-auth-settings API surface now that enabling auth lives in the
Feature Toggles system, and drop env_required now that /auth/me no longer
returns it. Add a current-user chip with a logout action to the sidebar
footer, shown only in enabled auth mode.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* refactor(auth): make features.auth the single on/off control

Per review feedback: authentication configuration now lives entirely in
the existing Feature Toggles system rather than a separate Authentication
settings section.

- The auth mode is derived from the features.auth runtime flag
  (KANDEV_FEATURES_AUTH): off -> disabled, on+no-admin -> setup,
  on+admin -> enabled. Removed the parallel auth.mode system setting,
  the SetEnabled service method, GET/PATCH /api/v1/auth/settings, the
  AuthConfig.Required field, KANDEV_AUTH_REQUIRED, and the auth service's
  Settings-store dependency and EnvRequired/env_required.
- The features.auth registry entry is re-described as the actual enable
  switch (high-risk, restart-required) and defaults off in every profile
  (dev was previously true — that only revealed UI before; now it would
  enforce, so it must be an explicit opt-in).
- Setup no longer persists a mode; completing the wizard creates the admin
  and the mode auto-derives to enabled.
- Startup 'exposed without auth' warning + all docs (public guide,
  run-as-a-service, spec, ADR, AGENTS.md) point at the feature toggle.
- E2E: auth specs use KANDEV_FEATURES_AUTH; screenshot spec captures the
  Feature Toggles page instead of the removed Authentication page; the
  member-restriction assertion targets an admin invite route.
- Tests updated across service/httpmw/httpapi/profiles for the derived
  mode; full suite green (only pre-existing sandbox-FS failures remain).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(auth): complete cross-user isolation — integrations, automation, plan/walkthrough, session reads

A follow-up isolation audit found that per-user scoping had been wired into
the task service, secrets, office, and the WS gateway — but several
workspace/task/session-scoped surfaces still resolved a caller-supplied ID
with no ownership gate. Closed all of them; cross-user access now returns
404 across the board.

- Integrations (jira/linear/sentry/slack/azure-devops/gitlab/github/
  workflow-sync): a global middleware authorizes the workspace_id (query, or
  gitlab's /workspaces/:id/ path) on those route groups. Previously a member
  who knew another user's workspace ID could read/overwrite their integration
  connection config and watches.
- Automation: workspace-authorizer seam gates get/list/create/update/delete/
  enable/disable, triggers, run history, and — critically — the webhook
  secret. Was fully cross-tenant over WS before.
- Task plans (PlanService) and walkthroughs (WalkthroughService): task-
  authorizer seam on every read/write. These bypassed the task service.
- Session reads that hit the repo directly — session turns, task context,
  git snapshots, file reviews — now call AuthorizeSession/TaskAccess.

Internal callers (pollers, schedulers, MCP) carry no identity and stay
unscoped; disabled mode is unaffected. Regression tests added
(plan/walkthrough scoping, automation authorizer matrix, integration path
extraction) and the two-user E2E now asserts integration routes are shielded.
Docs (spec, ADR, public guide) updated to state what is isolated and that
filesystem/agent-credential isolation remains an OS-level limitation.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(web): current-user chip uses border hover, not accent fill

The sidebar current-user chip used hover:bg-accent (a purpleish fill).
Switch to the border-hover idiom used elsewhere (border border-transparent
-> hover:border-border) with the neutral hover:bg-muted/50 the adjacent
workspace picker uses, so the hover reads as a subtle outline instead of a
colored background. Screenshot spec now captures the chip hover state.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(web): right-align current-user chip in the sidebar footer

Push the user chip to the far right of the expanded footer (ml-auto) so it
sits opposite the left-aligned icon buttons; collapsed mode keeps the
centered vertical stack unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(auth): reject synthetic identity on management routes and scope office run broadcasts

Address code-review findings on the opt-in authentication PR:

- P1: while auth is disabled the middleware injects a synthetic admin
  (RoleAdmin), and RequireAdmin alone let it through — so anyone hitting a
  not-yet-enabled instance could mint a PAT or plant an admin that survives
  enablement and hijacks first-run setup. Add authn.RequireRealIdentity and
  guard the credential/session/user-management routes with it (public
  bootstrap routes stay reachable). Responds 404 while disabled.
- P1: office run events carry task_id but no workspace_id, so they fell back
  to a global broadcast and leaked across users. Resolve the workspace from
  the task and route via the new Hub.BroadcastToWorkspaceOrDrop, which fails
  closed (drops rather than global-broadcasts) when the workspace is unknown
  and auth is enforced.
- nit: type createAuthSlice as a single-arg creator and call it with just set.

Also register the authentication docs page and its settings routes in
docs/public/coverage.json so the public-docs validator passes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Kandev Agent <agent@kandev.dev>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-25 00:10:22 +01:00

743 lines
28 KiB
Go

package lifecycle
import (
"context"
"errors"
"fmt"
"time"
"github.com/google/uuid"
"go.uber.org/zap"
"golang.org/x/sync/singleflight"
"github.com/kandev/kandev/internal/agent/runtime/activity"
"github.com/kandev/kandev/internal/agentctl/tracing"
"github.com/kandev/kandev/internal/common/appctx"
"github.com/kandev/kandev/internal/events"
"github.com/kandev/kandev/internal/secrets"
)
// ErrSessionWorkspaceNotReady indicates the task session exists but does not yet
// have a resolved workspace path (typically while worktree preparation is in progress).
var ErrSessionWorkspaceNotReady = errors.New("session workspace not ready")
// coalescedExecutionCreationTimeout matches the runtime's 60-second agentctl
// startup window while preventing blocked instance I/O from owning the shared
// session slot and its activity lease for the lifetime of the manager.
const coalescedExecutionCreationTimeout = time.Minute
// GetOrEnsureExecution returns an existing execution or creates one on-demand.
// Use this for workspace-oriented operations (files, shell, inference, ports, vscode, LSP)
// that should survive backend restarts. For operations requiring a running agent
// process (prompt, cancel, mode), use GetExecutionBySessionID instead.
//
// Concurrent calls for the same sessionID are deduplicated via singleflight.
func (m *Manager) GetOrEnsureExecution(ctx context.Context, sessionID string) (*AgentExecution, error) {
if sessionID == "" {
return nil, fmt.Errorf("session_id is required")
}
// Per-user workspace scoping (opt-in auth): user-facing session surfaces
// funnel through here; internal callers pass a ctx without an identity
// and are unaffected.
if check := m.sessionAccessCheck; check != nil {
if err := check(ctx, sessionID); err != nil {
return nil, err
}
}
// Fast path: execution already in memory
if execution, exists := m.executionStore.GetBySessionID(sessionID); exists {
return execution, nil
}
// Slow path: create on-demand, deduplicated by sessionID-keyed singleflight.
// Use ensureWorkspaceExecutionLocked (not EnsureWorkspaceExecutionForSession)
// to avoid recursing into the same singleflight slot we already hold.
value, err := m.doCoalescedExecution(ctx, sessionID, func(sharedCtx context.Context) (interface{}, error) {
return m.ensureWorkspaceExecutionLocked(sharedCtx, "", sessionID)
})
if err != nil {
return nil, err
}
return value.(*AgentExecution), nil
}
// GetOrEnsureExecutionForEnvironment returns an execution for a task environment,
// creating one on-demand from the workspace info provider when needed.
//
// Important: this MUST share the session-keyed singleflight bucket with
// GetOrEnsureExecution(sessionID) and EnsureWorkspaceExecutionForSession.
// A previous version keyed by `"env:" + envID`, which let a concurrent
// session-keyed call race past it (each path observed "no execution" for its
// own key, both called createExecution, both ExecutionStore.Add, the second
// silently overwrote the bySession index, and the first execution's
// agent subprocess was orphaned). See `ErrExecutionAlreadyExistsForSession`.
func (m *Manager) GetOrEnsureExecutionForEnvironment(ctx context.Context, taskEnvironmentID string) (*AgentExecution, error) {
if taskEnvironmentID == "" {
return nil, fmt.Errorf("task_environment_id is required")
}
// Per-user scoping (opt-in auth) — before the cache short-circuit so a
// cached execution cannot be reached by a non-owner.
if check := m.environmentAccessCheck; check != nil {
if err := check(ctx, taskEnvironmentID); err != nil {
return nil, err
}
}
if execution, exists := m.executionStore.GetByTaskEnvironmentID(taskEnvironmentID); exists {
return execution, nil
}
if m.workspaceInfoProvider == nil {
return nil, fmt.Errorf("workspace info provider not configured")
}
info, err := m.workspaceInfoProvider.GetWorkspaceInfoForEnvironment(ctx, taskEnvironmentID)
if err != nil {
return nil, fmt.Errorf("failed to get workspace info for environment %s: %w", taskEnvironmentID, err)
}
if info == nil {
return nil, fmt.Errorf("task environment %s not found", taskEnvironmentID)
}
if info.TaskEnvironmentID == "" {
return nil, fmt.Errorf("task environment %s has no task_environment_id", taskEnvironmentID)
}
if info.TaskEnvironmentID != taskEnvironmentID {
return nil, fmt.Errorf("workspace info resolved environment %s, want %s", info.TaskEnvironmentID, taskEnvironmentID)
}
if info.WorkspacePath == "" {
return nil, fmt.Errorf("%w: task environment %s has no workspace path yet", ErrSessionWorkspaceNotReady, taskEnvironmentID)
}
if info.SessionID == "" {
return nil, fmt.Errorf("task environment %s has no task session", taskEnvironmentID)
}
// Share the sessionID-keyed bucket so we deduplicate against any concurrent
// GetOrEnsureExecution(sessionID) / EnsureWorkspaceExecutionForSession for
// the same session.
value, err := m.doCoalescedExecution(ctx, info.SessionID, func(sharedCtx context.Context) (interface{}, error) {
if execution, exists := m.executionStore.GetBySessionID(info.SessionID); exists {
return execution, nil
}
if execution, exists := m.executionStore.GetByTaskEnvironmentID(taskEnvironmentID); exists {
return execution, nil
}
// createExecution publishes AgentctlStarting before spawning the
// waitForAgentctlReady goroutine, so frontend gates flip out of
// `undefined` even on this lazy-create path.
execution, err := m.createExecution(sharedCtx, info.TaskID, info)
if err != nil {
return nil, err
}
return execution, nil
})
if err != nil {
return nil, err
}
return value.(*AgentExecution), nil
}
// EnsureWorkspaceExecutionForSession ensures an agentctl execution exists for a specific task session.
// This is used when the frontend provides a session ID (e.g., from URL path /task/[id]/[sessionId]).
// If an execution already exists for the session, it returns it. Otherwise, it creates a new execution
// using the session's workspace configuration from the database.
//
// Concurrent calls (including from GetOrEnsureExecution and
// GetOrEnsureExecutionForEnvironment) are deduplicated via the same
// sessionID-keyed singleflight bucket so they cannot race past their
// individual check-then-act guards and create duplicate executions.
func (m *Manager) EnsureWorkspaceExecutionForSession(ctx context.Context, taskID, sessionID string) (*AgentExecution, error) {
if sessionID == "" {
return nil, fmt.Errorf("session_id is required")
}
// Fast path: execution already in memory
if execution, exists := m.executionStore.GetBySessionID(sessionID); exists {
return execution, nil
}
value, err := m.doCoalescedExecution(ctx, sessionID, func(sharedCtx context.Context) (interface{}, error) {
return m.ensureWorkspaceExecutionLocked(sharedCtx, taskID, sessionID)
})
if err != nil {
return nil, err
}
return value.(*AgentExecution), nil
}
func (m *Manager) doCoalescedExecution(
ctx context.Context,
key string,
operation func(context.Context) (interface{}, error),
) (interface{}, error) {
result := m.ensureExecutionGroup.DoChan(key, func() (interface{}, error) {
sharedCtx, cancel := m.coalescedExecutionContext(ctx)
defer cancel()
return operation(sharedCtx)
})
return awaitCoalescedResult(ctx, result)
}
func (m *Manager) coalescedExecutionContext(ctx context.Context) (context.Context, context.CancelFunc) {
sharedCtx, cancel := context.WithTimeout(context.WithoutCancel(ctx), coalescedExecutionCreationTimeout)
if m.stopCh == nil {
return sharedCtx, cancel
}
go func() {
select {
case <-m.stopCh:
cancel()
case <-sharedCtx.Done():
}
}()
return sharedCtx, cancel
}
func awaitCoalescedResult(
ctx context.Context,
result <-chan singleflight.Result,
) (interface{}, error) {
if err := ctx.Err(); err != nil {
return nil, err
}
select {
case <-ctx.Done():
return nil, ctx.Err()
case completed := <-result:
if err := ctx.Err(); err != nil {
return nil, err
}
if completed.Err != nil {
return nil, completed.Err
}
return completed.Val, nil
}
}
// ensureWorkspaceExecutionLocked is the body of EnsureWorkspaceExecutionForSession
// run inside the sessionID-keyed singleflight bucket. Callers other than
// EnsureWorkspaceExecutionForSession must already hold the singleflight slot.
func (m *Manager) ensureWorkspaceExecutionLocked(ctx context.Context, taskID, sessionID string) (*AgentExecution, error) {
// Double-check after acquiring the slot — a peer in the same group may have
// finished while we were waiting.
if execution, exists := m.executionStore.GetBySessionID(sessionID); exists {
return execution, nil
}
if m.workspaceInfoProvider == nil {
return nil, fmt.Errorf("workspace info provider not configured")
}
info, err := m.workspaceInfoProvider.GetWorkspaceInfoForSession(ctx, taskID, sessionID)
if err != nil {
return nil, fmt.Errorf("failed to get workspace info for session %s: %w", sessionID, err)
}
if info == nil {
return nil, fmt.Errorf("session %s not found", sessionID)
}
// Resolve taskID from provider when caller doesn't have it (e.g., GetOrEnsureExecution)
if taskID == "" {
taskID = info.TaskID
}
if info.TaskEnvironmentID != "" {
if execution, exists := m.executionStore.GetByTaskEnvironmentID(info.TaskEnvironmentID); exists {
m.logger.Info("reusing existing execution for task environment",
zap.String("task_id", taskID),
zap.String("session_id", sessionID),
zap.String("task_environment_id", info.TaskEnvironmentID),
zap.String("execution_id", execution.ID))
return execution, nil
}
}
if info.WorkspacePath == "" {
return nil, fmt.Errorf("%w: session %s has no workspace path yet", ErrSessionWorkspaceNotReady, sessionID)
}
m.logger.Info("creating execution for task session",
zap.String("task_id", taskID),
zap.String("session_id", sessionID),
zap.String("workspace_path", info.WorkspacePath),
zap.String("acp_session_id", info.ACPSessionID))
// createExecution publishes AgentctlStarting before spawning the
// waitForAgentctlReady goroutine, so workspace-only executions also
// notify the frontend without racing the readiness event.
execution, err := m.createExecution(ctx, taskID, info)
if err != nil {
return nil, err
}
// For workspace-only executions (no agent), wait for agentctl to be ready
// then connect the workspace stream so process output can be received.
// Note: AgentctlReady/Error events are already handled by waitForAgentctlReady
// (started by createExecution), so this goroutine only connects the stream.
go func() {
// Use detached context that respects stopCh for graceful shutdown
waitCtx, cancel := appctx.Detached(ctx, m.stopCh, 60*time.Second)
defer cancel()
if err := execution.agentctl.WaitForReady(waitCtx, 60*time.Second); err != nil {
m.logger.Error("agentctl not ready for workspace stream connection",
zap.String("execution_id", execution.ID),
zap.Error(err))
return
}
// Connect workspace stream for process output (agent stream not needed for workspace-only)
if m.streamManager != nil {
m.logger.Info("connecting workspace stream for workspace-only execution",
zap.String("execution_id", execution.ID))
m.streamManager.ConnectWorkspaceStream(execution, nil)
}
}()
return execution, nil
}
// GetExecutionIDForSession returns the execution ID for a session from the in-memory
// execution store. Returns empty string and error if no execution is found.
func (m *Manager) GetExecutionIDForSession(_ context.Context, sessionID string) (string, error) {
if execution, exists := m.executionStore.GetBySessionID(sessionID); exists {
return execution.ID, nil
}
return "", fmt.Errorf("%w: %s", ErrNoExecutionForSession, sessionID)
}
// IsAgentCommandConfigured reports whether an execution has been promoted from
// workspace-only infrastructure to an agent execution ready to start.
func (m *Manager) IsAgentCommandConfigured(executionID string) bool {
configured := false
_ = m.executionStore.WithRLock(executionID, func(execution *AgentExecution) {
configured = execution.AgentCommand != ""
})
return configured
}
// EnsurePassthroughExecution ensures an execution exists for a passthrough session
// and starts the passthrough process if needed. This is called when the terminal
// handler receives a connection for a session that might need recovery after backend restart.
//
// The sessionID is required. If taskID is empty, it will be looked up from:
// 1. The existing execution (if any)
// 2. The workspace info provider
//
// Returns the execution with a running passthrough process, or an error.
func (m *Manager) EnsurePassthroughExecution(ctx context.Context, sessionID string) (*AgentExecution, error) {
// Per-user scoping (opt-in auth) — before the cache short-circuit so a
// cached execution cannot be reached by a non-owner.
if check := m.sessionAccessCheck; check != nil {
if err := check(ctx, sessionID); err != nil {
return nil, err
}
}
// Check if execution already exists with a running passthrough process.
// PassthroughProcessID is not cleared on exit, so a stale ID can point at
// a dead process; verify the runner still has it before short-circuiting,
// otherwise a fast-failed resume launch would keep returning the dead ID
// and the WS handler's IsProcessReadyOrPending check would 503 forever.
if execution, exists := m.executionStore.GetBySessionID(sessionID); exists {
if execution.PassthroughProcessID != "" {
if runner := m.GetInteractiveRunner(); runner != nil && runner.IsProcessReadyOrPending(execution.PassthroughProcessID) {
return execution, nil
}
m.logger.Info("execution has stale passthrough process ID, relaunching",
zap.String("session_id", sessionID),
zap.String("execution_id", execution.ID),
zap.String("stale_process_id", execution.PassthroughProcessID))
}
return m.resumeExistingExecution(ctx, sessionID, execution)
}
// No execution exists - need to create one from session info
return m.createExecutionFromSessionInfo(ctx, sessionID)
}
// resumeExistingExecution starts the passthrough process for an existing execution
// that has no running process (e.g., after backend restart).
func (m *Manager) resumeExistingExecution(ctx context.Context, sessionID string, execution *AgentExecution) (*AgentExecution, error) {
m.logger.Info("execution exists but passthrough process not running, starting",
zap.String("session_id", sessionID),
zap.String("execution_id", execution.ID))
if err := m.ResumePassthroughSession(ctx, sessionID); err != nil {
return nil, fmt.Errorf("resume passthrough session %s: %w", sessionID, err)
}
// Get updated execution with process ID
execution, exists := m.executionStore.GetBySessionID(sessionID)
if !exists {
return nil, fmt.Errorf("execution disappeared after resuming passthrough session %s", sessionID)
}
return execution, nil
}
// createExecutionFromSessionInfo creates a new execution for a passthrough session
// when no execution exists (e.g., backend restarted and execution store was cleared).
func (m *Manager) createExecutionFromSessionInfo(ctx context.Context, sessionID string) (*AgentExecution, error) {
if m.workspaceInfoProvider == nil {
return nil, fmt.Errorf("cannot restore session %s: workspace info provider not configured", sessionID)
}
// Get workspace info from the provider (looks up session to get taskID, workspace path, etc.)
info, err := m.workspaceInfoProvider.GetWorkspaceInfoForSession(ctx, "", sessionID)
if err != nil {
return nil, fmt.Errorf("get workspace info for session %s: %w", sessionID, err)
}
if info.WorkspacePath == "" {
return nil, fmt.Errorf("%w: session %s has no workspace path configured", ErrSessionWorkspaceNotReady, sessionID)
}
if info.TaskID == "" {
return nil, fmt.Errorf("session %s has no associated task ID", sessionID)
}
// Verify this session should use passthrough mode
if err := m.verifyPassthroughEnabled(ctx, sessionID, workspaceExecutionProfileID(info)); err != nil {
return nil, err
}
// If agent ID not in workspace info (snapshot missing/empty), resolve from profile
executionProfileID := workspaceExecutionProfileID(info)
if info.AgentID == "" && executionProfileID != "" && m.profileResolver != nil {
profileInfo, err := m.profileResolver.ResolveProfile(ctx, executionProfileID)
if err != nil {
return nil, fmt.Errorf("resolve agent for session %s: %w", sessionID, err)
}
info.AgentID = profileInfo.AgentName
}
// Create the execution
m.logger.Info("creating execution for passthrough session",
zap.String("task_id", info.TaskID),
zap.String("session_id", sessionID),
zap.String("workspace_path", info.WorkspacePath))
execution, err := m.createExecution(ctx, info.TaskID, info)
if err != nil {
return nil, fmt.Errorf("create execution for session %s: %w", sessionID, err)
}
// Start the passthrough process using resume command (recovery after restart)
m.logger.Info("starting passthrough process for session",
zap.String("session_id", sessionID),
zap.String("execution_id", execution.ID))
if err := m.ResumePassthroughSession(ctx, sessionID); err != nil {
return nil, fmt.Errorf("start passthrough process for session %s: %w", sessionID, err)
}
// Get updated execution with process ID
execution, exists := m.executionStore.GetBySessionID(sessionID)
if !exists {
return nil, fmt.Errorf("execution disappeared after starting passthrough session %s", sessionID)
}
return execution, nil
}
// verifyPassthroughEnabled checks if the session's profile has CLI passthrough enabled.
func (m *Manager) verifyPassthroughEnabled(ctx context.Context, sessionID, profileID string) error {
if m.profileResolver == nil || profileID == "" {
return fmt.Errorf("session %s has no profile configured for passthrough mode", sessionID)
}
profileInfo, err := m.profileResolver.ResolveProfile(ctx, profileID)
if err != nil {
m.logger.Warn("failed to resolve profile for passthrough check",
zap.String("session_id", sessionID),
zap.String("profile_id", profileID),
zap.Error(err))
return fmt.Errorf("session %s: failed to resolve profile %s: %w", sessionID, profileID, err)
}
if profileInfo == nil || !profileInfo.CLIPassthrough {
return fmt.Errorf("session %s is not configured for CLI passthrough mode", sessionID)
}
return nil
}
// createExecution creates an agentctl execution.
// The agent subprocess is NOT started - call ConfigureAgent + Start explicitly.
func (m *Manager) createExecution(ctx context.Context, taskID string, info *WorkspaceInfo) (*AgentExecution, error) {
activityLease, err := m.acquireActivity(ctx, activity.KindExecutionStarting)
if err != nil {
return nil, err
}
defer activityLease.Release()
activityLease.SetKind(activity.KindExecutionPreparing)
// Select runtime based on executor type; falls back to standalone if empty/unavailable
rt, err := m.getExecutorBackend(info.ExecutorType)
if err != nil {
return nil, fmt.Errorf("no runtime configured: %w", err)
}
if info.AgentID == "" {
return nil, fmt.Errorf("agent ID is required in WorkspaceInfo")
}
executionID := uuid.New().String()
agentConfig, ok := m.registry.Get(info.AgentID)
if !ok {
return nil, fmt.Errorf("agent type %q not found in registry", info.AgentID)
}
// Forward AgentProfile.EnvVars to the runtime instance. The Launch /
// ResumeSession paths merge these into req.Env via buildEnvForExecution
// before calling LaunchAgent; the lazy workspace-only path (any
// GetOrEnsureExecution* caller after backend restart) lands here directly,
// so without this merge the runtime instance gets spawned with empty env
// and CLAUDE_CONFIG_DIR (and any other workspace profile var) is lost.
// The agent subprocess inherits the instance env via agentctl, and ACP
// session/load then looks under the wrong SDK root → -32002 Resource not
// found.
env := map[string]string{}
var profileInfo *AgentProfileInfo
executionProfileID := workspaceExecutionProfileID(info)
if executionProfileID != "" && m.profileResolver != nil {
resolvedProfile, err := m.profileResolver.ResolveProfile(ctx, executionProfileID)
if err != nil {
m.logger.Warn("failed to resolve profile for workspace execution",
zap.String("execution_profile_id", executionProfileID),
zap.Error(err))
} else {
profileInfo = resolvedProfile
}
}
m.mergeAgentProfileEnvFromInfo(ctx, profileInfo, env)
managedReq := &LaunchRequest{ExecutorType: info.ExecutorType, Env: env}
if err := m.prepareManagedGoCacheEnvironment(ctx, managedReq); err != nil {
return nil, err
}
env = managedReq.Env
autoApprove := false
var autoApproveOverride *bool
if profileInfo != nil {
autoApprove = profileInfo.AutoApprove
autoApproveOverride = boolPtr(profileInfo.AutoApprove)
}
if len(env) == 0 {
env = nil
}
metadata := make(map[string]interface{}, len(info.Metadata)+1)
for key, value := range info.Metadata {
metadata[key] = value
}
if managedReq.managedGoCachePath != "" {
metadata[managedGoCacheMetadataKey] = managedReq.managedGoCachePath
}
req := &ExecutorCreateRequest{
InstanceID: executionID,
TaskID: taskID,
SessionID: info.SessionID,
TaskEnvironmentID: info.TaskEnvironmentID,
AgentProfileID: executionProfileID,
OfficeAgentProfileID: info.AgentProfileID,
WorkspacePath: info.WorkspacePath,
Protocol: string(agentConfig.Runtime().Protocol),
Env: env,
AutoApprovePermissions: autoApprove,
AutoApprovePermissionsOverride: autoApproveOverride,
AgentConfig: agentConfig,
Metadata: metadata,
PreviousExecutionID: info.AgentExecutionID,
AuthToken: m.revealRuntimeSecret(ctx, info.Metadata, MetadataKeyAuthTokenSecret),
BootstrapNonce: m.revealRuntimeSecret(ctx, info.Metadata, MetadataKeyBootstrapNonceSecret),
}
runtimeInstance, err := rt.CreateInstance(ctx, req)
if err != nil {
return nil, fmt.Errorf("failed to create execution: %w", err)
}
execution := runtimeInstance.ToAgentExecution(req)
execution.RuntimeName = rt.Name()
// Cache the resolved profile env on the execution so a subsequent
// configureAndStartAgent (when this workspace-only execution is promoted)
// reuses it via mergeAgentProfileEnvForExecution instead of doing another
// secret-store round-trip. Mirrors the Launch path (manager_launch.go).
if env != nil {
m.cacheResolvedProfileEnv(execution, env)
}
// Set the ACP session ID for session resumption
if info.ACPSessionID != "" {
execution.ACPSessionID = info.ACPSessionID
}
// Create trace span for workspace-only execution
_, sessionSpan := tracing.TraceSessionStart(
context.Background(), taskID, info.SessionID, executionID,
)
execution.SetSessionSpan(sessionSpan)
if execution.agentctl != nil {
execution.agentctl.SetTraceContext(execution.SessionTraceContext())
}
if addErr := m.executionStore.Add(execution); addErr != nil {
// Lost a race: another path created an execution for this session
// between our check and our Add. Roll back the runtime instance we
// just spawned (otherwise its subprocess is orphaned) and return the
// winner so the caller observes a single execution per session.
if errors.Is(addErr, ErrExecutionAlreadyExistsForSession) {
m.rollbackRacedExecution(ctx, rt, runtimeInstance, execution)
if existing, ok := m.executionStore.GetBySessionID(info.SessionID); ok {
return existing, nil
}
}
return nil, fmt.Errorf("failed to register execution: %w", addErr)
}
// Persist executors_running row in lockstep with the in-memory Add so the
// DB never holds an execution_id the store doesn't know about. This is the
// structural fix for the divergence bug — pre-refactor, the orchestrator
// wrote the row later via a full-row UPDATE that could race with the store.
m.persistExecutorRunning(ctx, execution)
// Persist agentctl auth token only after the execution is tracked, so a
// race-lost rollback never leaves an orphaned secret in the store.
m.persistRuntimeSecrets(ctx, runtimeInstance, execution)
go m.pollOneRemoteStatus(context.Background(), execution)
// Publish Starting BEFORE spawning waitForAgentctlReady so subscribers
// always observe Starting → Ready/Error in order. Doing it after the go
// call would race: if Health succeeds before this line runs, Ready could
// be published first and the frontend gate would briefly flicker.
m.eventPublisher.PublishAgentctlEvent(ctx, events.AgentctlStarting, execution, "")
go m.waitForAgentctlReady(execution)
m.logger.Info("execution created",
zap.String("execution_id", executionID),
zap.String("task_id", taskID),
zap.String("workspace_path", info.WorkspacePath),
zap.Stringer("runtime", execution.RuntimeName))
return execution, nil
}
func workspaceExecutionProfileID(info *WorkspaceInfo) string {
if info == nil {
return ""
}
if info.ExecutionProfileID != "" {
return info.ExecutionProfileID
}
return info.AgentProfileID
}
// rollbackRacedExecution tears down an execution that lost a session-conflict
// race in the store. Without this the runtime instance (agentctl + agent
// subprocess if any) keeps running with no tracking entry, and no cleanup path
// will ever find it.
func (m *Manager) rollbackRacedExecution(ctx context.Context, rt ExecutorBackend, runtimeInstance *ExecutorInstance, execution *AgentExecution) {
m.logger.Warn("rolling back duplicate execution after session-conflict race",
zap.String("execution_id", execution.ID),
zap.String("session_id", execution.SessionID))
if rt != nil && runtimeInstance != nil {
if stopErr := rt.StopInstance(ctx, runtimeInstance, true); stopErr != nil {
m.logger.Warn("failed to stop raced runtime instance during rollback",
zap.String("execution_id", execution.ID),
zap.Error(stopErr))
}
}
if execution.agentctl != nil {
execution.agentctl.Close()
}
execution.EndSessionSpan()
}
const (
// MetadataKeyAuthTokenSecret is the metadata key for the encrypted agentctl auth token secret ID.
MetadataKeyAuthTokenSecret = "env_secret_id_AGENTCTL_AUTH_TOKEN"
// MetadataKeyBootstrapNonceSecret stores the encrypted Docker bootstrap nonce.
// It lets the backend re-handshake after a container restart starts a new
// agentctl process with a fresh auth token.
MetadataKeyBootstrapNonceSecret = "env_secret_id_AGENTCTL_BOOTSTRAP_NONCE"
)
func (m *Manager) persistRuntimeSecrets(ctx context.Context, instance *ExecutorInstance, execution *AgentExecution) {
m.persistAuthToken(ctx, instance, execution)
m.persistBootstrapNonce(ctx, instance, execution)
}
// persistAuthToken stores the agentctl handshake auth token in SecretStore
// and saves the secret ID in the execution's metadata for recovery after restart.
func (m *Manager) persistAuthToken(ctx context.Context, instance *ExecutorInstance, execution *AgentExecution) {
m.persistRuntimeSecret(ctx, instance, execution, MetadataKeyAuthTokenSecret, "agentctl-auth", instance.AuthToken)
}
func (m *Manager) persistBootstrapNonce(ctx context.Context, instance *ExecutorInstance, execution *AgentExecution) {
m.persistRuntimeSecret(ctx, instance, execution, MetadataKeyBootstrapNonceSecret, "agentctl-bootstrap", instance.BootstrapNonce)
}
func (m *Manager) persistRuntimeSecret(
ctx context.Context,
instance *ExecutorInstance,
execution *AgentExecution,
metadataKey string,
secretNamePrefix string,
value string,
) {
if value == "" || m.secretStore == nil {
return
}
secret := &secrets.SecretWithValue{
Secret: secrets.Secret{
Name: fmt.Sprintf("%s-%s", secretNamePrefix, truncateID(instance.InstanceID, 12)),
},
Value: value,
}
if err := m.secretStore.Create(ctx, secret); err != nil {
m.logger.Error("failed to persist runtime secret",
zap.String("instance_id", instance.InstanceID),
zap.String("metadata_key", metadataKey),
zap.Error(err))
return
}
if execution.Metadata == nil {
execution.Metadata = make(map[string]interface{})
}
execution.Metadata[metadataKey] = secret.ID
m.logger.Debug("persisted runtime secret in secret store",
zap.String("instance_id", instance.InstanceID),
zap.String("metadata_key", metadataKey),
zap.String("secret_id", secret.ID))
}
func (m *Manager) revealRuntimeSecret(ctx context.Context, metadata map[string]interface{}, metadataKey string) string {
if m.secretStore == nil {
return ""
}
secretID := getMetadataString(metadata, metadataKey)
if secretID == "" {
return ""
}
value, err := m.secretStore.Reveal(ctx, secretID)
if err != nil {
m.logger.Warn("failed to reveal runtime secret",
zap.String("metadata_key", metadataKey),
zap.String("secret_id", secretID),
zap.Error(err))
return ""
}
return value
}
// truncateID safely truncates an ID string to maxLen characters.
func truncateID(id string, maxLen int) string {
if len(id) <= maxLen {
return id
}
return id[:maxLen]
}