kandev/docs/specs/kubernetes-executor/spec.md

status, created, updated, owner

status	created	updated	owner
implemented	2026-08-24	2026-09-01	Kandev

Kubernetes Executor

Why

Teams that already operate Kubernetes need Kandev sessions to run inside their cluster policies, networks, storage, and workload identity boundary. Today they must use a host-local, Docker, Sprites, or SSH executor instead of scheduling a first-class Pod with an administrator-reviewed workload template.

What

• Administrators can create a Kubernetes executor for one API server context and one namespace by using either an explicit kubeconfig file on the Kandev host or Kandev's in-cluster service-account credentials.
• Administrators can test authentication, API discovery, namespace access, required RBAC subresources, and optional Pod/PVC admission before selecting the executor for work.
• Administrators can create Kubernetes profiles from one strict core/v1 PodTemplate YAML document. The profile chooses the main container, Linux architecture, and workspace storage policy.
• One task session maps to one Pod. Kandev starts its existing agentctl control process in the designated container and reaches it through a loopback-only Kubernetes port-forward.
• The template can define images, sidecars, init containers, resource and security settings, scheduling policy, image-pull secrets, and workload service accounts. Kandev retains exclusive control of the fields needed for runtime identity, bootstrap, transport, workspace storage, recovery, and cleanup. Ordinary non-main containers may explicitly request one RO/RW kandev-workspace mount at exactly /workspace; admission must preserve its recipient and shape. Runtime/auth mounts and init/ephemeral containers remain excluded. See workspace grants.
• Profiles support a Kandev-managed per-session PVC, a Pod-scoped emptyDir, or an existing namespaced PVC. Kandev never deletes an existing PVC.
• Ordinary Stop and backend shutdown preserve the Pod and workspace for resume. Archive, delete, and explicit force cleanup remove only resources proven to belong to the recorded session.
• Members can discover and use administrator-configured Kubernetes profiles. They cannot create, edit, delete, or test Kubernetes executor/profile configuration.
• Linux amd64 and arm64 are supported. The profile fixes the platform and Kandev ensures the Pod is scheduled to a matching node before injecting the matching helper binary.
• The settings experience exposes the same capabilities on desktop and mobile, with touch-visible actions and no page-level horizontal overflow.
• Kubernetes is represented by a package-style Pod glyph wherever an executor or runtime icon is shown. It does not reuse the generic cloud-computing or plain cube glyph.
• A Kubernetes task's executor disclosure shows the exact authorized session Pod status rather than falling back to the recorded task-environment ready value. It exposes the Pod identity, phase/container state, restart count, workspace mode, and sanitized failure reason when present.
• The task disclosure provides compact icon-only Refresh and Profile settings actions in its summary header. The controls remain keyboard accessible, labelled, and at least 44 px on touch layouts without becoming the visual focus of the disclosure. On fine-pointer desktop it may use the existing compact popover; on a coarse pointer it is a named button that opens a Drawer with the same information and actions. Kubernetes does not expose the generic Reset environment action, because Pod/PVC cleanup remains owned by the session Stop/Resume and terminal Archive/Delete lifecycle.
• The Kubernetes task disclosure presents one deliberate Pod summary: a Pod-shaped identity header, a human-readable live status, grouped runtime facts, and quiet corner controls for Refresh and Profile settings. Technical identities use monospace typography; state, dates, labels, and actions do not.
• Refreshing a task disclosure always has visible in-progress feedback. A click made while background polling already owns the same request joins that request instead of appearing to succeed without waiting or issuing a duplicate read.
• Every Kubernetes profile page begins with the shared executor's connection configuration, connection and permissions test, and executor-wide active sessions before profile details, workload, workspace, credentials, or scripts. Connection fields are edited inline on that profile page instead of through a nested connection destination. The page states that connection values are shared by every profile under the executor. Testing uses the current unsaved connection, workload, and workspace values.
• Kubernetes connection settings and profile settings remain separate persisted resources, but one profile page coordinates their dirty state, validation, active-session confirmation, save, and reset behavior as one settings hierarchy. A partial API failure leaves the successfully saved resource clean and the failed resource visibly dirty for retry.
• The settings tree treats a configured Kubernetes executor as a profile group, not as a second connection subpage. Task disclosure settings links target the exact executor_profile_id. Legacy executor-level URLs redirect to a profile when one exists and remain only as an orphan-executor recovery surface when no profile exists.
• Sidebar, Kanban, and task-page Kubernetes status surfaces converge on the same current credential view. Rotating or repairing kubeconfig credentials must not leave list/card chrome pinned to a launch-time Unauthorized result after the current executor configuration can inspect the recorded Pod successfully.
• Compact sidebar and Kanban indicator hydration and responsive disclosure are owned by task executor status indicators and their system design.
• The raw PodTemplate YAML field grows and shrinks with its content from a compact minimum height. It has no independent vertical scrollbar; the settings page is the vertical scroll owner, while unwrapped YAML retains contained horizontal scrolling without widening the document.

Decision: ADR-2026-08-24-kubernetes-executor-resource-ownership.

Data model

No new database table is required. Existing entities carry the contract:

executors

• type: "k8s".
• config.auth_mode: "kubeconfig" | "in_cluster".
• config.kubeconfig_path: absolute path, required only for kubeconfig.
• config.kube_context: optional context name.
• config.namespace: required namespace; profile/task metadata cannot override it.
• config.request_timeout_seconds: bounded positive timeout.

Kubeconfig bytes and resolved credentials are never copied into this JSON.

executor_profiles

• config.platform: "linux/amd64" | "linux/arm64".
• config.main_container: container name, default kandev-agent.
• config.pod_template_yaml: one strict v1/PodTemplate YAML document, limited to 256 KiB.
• config.workspace.mode: "managed_pvc" | "empty_dir" | "existing_claim".
• config.workspace.size, storage_class, and access_modes: managed-PVC settings.
• config.workspace.claim_name: required for existing_claim.

executors_running.metadata

The authoritative runtime inventory records namespace, Pod name and UID, main container, platform, workspace mode, PVC name and UID, whether Kandev created the PVC, agentctl remote port, sanitized template/config hashes, and the validated workload profile snapshot used for that launch: Pod template, platform, main container, and storage configuration only. Kandev does not add resolved credentials, resolved profile environment values, injected files, or scripts to the snapshot; administrators remain responsible for literal values they place in the Pod template itself. The local port-forward port is process-local and is not durable.

API surface

Existing executor/profile CRUD routes accept type: "k8s" and the configuration above. Validation errors identify a stable field path.

POST /api/v1/kubernetes/test

Administrator-only. Accepts unsaved executor config and an optional profile config. Returns success, server_version, namespace, steps, warnings, and an optional sanitized error. Each step contains a stable key, success state, duration, and human-readable detail. Dry-run Pod/PVC admission may be performed when profile config is supplied; no retained workload is created.

GET /api/v1/kubernetes/executors/:id/sessions

Returns sanitized active session rows derived from executors_running, then verified against exact recorded Pods. It does not enumerate arbitrary namespace workloads. Rows include task/session IDs, Pod name/phase, container state/restarts, workspace kind, created time, and sanitized failure reason. Callers may supply task_id and, with it, an optional session_id to select only the authorized task/session rows needed by task chrome. Filtering happens before any Pod lookup, so refreshing one task disclosure never probes every Pod using the executor. A session filter without its task identity is invalid.

GET /api/v1/kubernetes/executors/:id/session-impact

Administrator-only. Returns an authoritative active_session_count for save confirmation by counting every Kubernetes executors_running row with the exact executor ID. Unlike the detailed session list, this count is not filtered through task ownership, inventory completeness, or live Pod status, so an administrator cannot unknowingly change shared reconnect or workload settings while another user's retained session is hidden. It returns no task, session, or cluster-resource identity.

State machine

1. Configured: the executor/profile are persisted but no cluster resource exists.
2. Provisioning storage: Kandev verifies an existing claim or creates a managed PVC.
3. Scheduling: Kandev creates the merged Pod and waits for the main container to run.
4. Bootstrapping: Kandev uploads helper/config/credentials through pods/exec, executes the resolved preparation script within its setup budget, signals the managed entrypoint only after preparation succeeds, and establishes port-forward. Preparation failure returns a bounded sanitized diagnostic without releasing agentctl.
5. Running: the existing nonce handshake succeeds and agentctl owns the agent subprocess.
6. Preserved: ordinary stop or backend shutdown closes local forwarding but retains Pod/workspace for resume.
7. Cleaning: terminal/forced cleanup validates exact identities, deletes the Pod, then deletes only a Kandev-created PVC.
8. Cleaned: resources are absent and runtime inventory can be removed.

A failed transition cleans only resources created by that attempt. A failed identity check or inventory lookup stops cleanup without deleting anything.

Permissions

• Administrators can create, edit, delete, validate, and test Kubernetes executors and profiles.
• Members can list executors/profiles, select an enabled profile, start tasks, resume their sessions, and view sanitized session status.
• The Kubernetes identity requires namespaced Pod and streaming-subresource access. PVC permissions are conditional on workspace mode. Optional Event or Log read access improves diagnostics but is not required to launch.
• The workload Pod's service account is separate from Kandev's API identity. Token automount defaults off when the template does not explicitly opt in.

Failure modes

• Missing/invalid Kubernetes configuration fails closed; Kandev never falls back to the standalone executor.
• Unknown template fields, multiple YAML documents, reserved-field conflicts, platform conflicts, or missing main-container image fail before an API write.
• Kubernetes admission, quota, scheduling, image-pull, PVC-binding, exec, and port-forward failures surface the causal step and a sanitized error.
• If an admission webhook mutates a Kandev-owned invariant, Kandev rejects the returned Pod and removes that exact newly created object.
• Every managed resource create carries a fresh Kandev-owned 256-bit nonce. After an ambiguous create result, Kandev accepts an exact-name lookup only if the admitted object preserves that request's nonce and every other owned field; otherwise it performs no checkpoint, bootstrap, or deletion.
• A proxy that cannot carry Kubernetes streaming subresources is reported as incompatible even when API discovery succeeds.
• Recovery rejects a Pod or PVC whose UID or required ownership labels differ from executors_running.
• If a Pod disappears while a managed PVC remains, resume can create a replacement Pod against the verified PVC. Lost emptyDir workspace is reported as unrecoverable.
• Cleanup is idempotent for NotFound but fails closed on identity mismatch, inventory failure, or ambiguous ownership.

Persistence guarantees

• Pod/PVC identities, auth token references, and bootstrap nonce references survive backend restart through existing executors_running metadata and runtime secret storage.
• A local port-forward never survives process restart; recovery creates a new loopback forward.
• A main-container restart preserves the Pod's runtime/auth emptyDir data and starts a new agentctl process. That process generates a new token and accepts one handshake with the persisted bootstrap nonce.
• Managed PVCs and existing claims preserve workspace data across Pod replacement. emptyDir preserves data only for the lifetime of its Pod.
• Workload-profile edits (Pod template, platform, main container, and storage) affect new sessions only. Running Pods retain their created spec and recorded identities; recovery that must replace a missing Pod uses the recorded workload launch snapshot rather than the profile's current contents.

Scenarios

• GIVEN valid kubeconfig credentials and namespace RBAC, WHEN an administrator tests the executor, THEN every mandatory permission and the API server version are shown without a retained workload.
• GIVEN Kandev runs in a Pod with the documented service account, WHEN an administrator selects in-cluster auth, THEN connection testing and task launch use that service-account identity without a kubeconfig file.
• GIVEN a valid profile with a sidecar, affinity, resource limits, and workload service account, WHEN a task starts, THEN those operator fields remain in the created Pod while Kandev-owned bootstrap fields match the documented invariants.
• GIVEN a profile attempts to replace a reserved command, mount, env key, identity label, namespace, or architecture selector, WHEN it is saved or tested, THEN validation identifies the conflicting field before creating a Pod.
• GIVEN a member can see a configured Kubernetes profile, WHEN they start a task, THEN the task may use the profile, but Kubernetes settings remain read-only and mutation APIs return forbidden.
• GIVEN a running Kubernetes session, WHEN the backend restarts, THEN recovery verifies the recorded Pod UID/labels, creates a new local forward, and reconnects without losing the workspace.
• GIVEN a running main container, WHEN Kubernetes restarts it, THEN agentctl starts again from the Pod runtime volume and Kandev re-handshakes with a newly generated token.
• GIVEN a running session, WHEN the user stops and later resumes it, THEN its Pod and workspace remain available and no terminal cleanup ran.
• GIVEN a terminally archived session using a managed PVC, WHEN cleanup runs, THEN Kandev deletes only the exact recorded Pod and PVC after UID and ownership-label validation.
• GIVEN a session uses an existing claim, WHEN the session is deleted, THEN Kandev deletes the Pod but leaves the claim intact.
• GIVEN a same-name object has a different UID or ownership labels, WHEN recovery or cleanup runs, THEN Kandev fails closed and leaves the object untouched.
• GIVEN a create result is ambiguous and a same-name object copies the expected labels and specification but lacks the exact per-request create nonce, WHEN Kandev reconciles the request, THEN it does not adopt, bootstrap, checkpoint, or delete that object.
• GIVEN retained Kubernetes inventory belongs to several users or contains a malformed row, WHEN an administrator saves executor or workload settings, THEN the confirmation count includes every exact executor row without exposing another user's session identity.
• GIVEN the settings page is opened on a phone, WHEN an administrator configures auth, template, and storage, THEN all required actions remain touch-visible, scroll within one vertical owner, and produce no document horizontal overflow.
• GIVEN an administrator opens a Kubernetes profile from the Executors tree, WHEN the profile page renders, THEN editable shared connection fields, cluster diagnostics, and executor-wide active sessions are visible before the editable profile sections without navigating to another settings page.
• GIVEN a member opens that Kubernetes profile, WHEN cluster status renders, THEN active sessions and the inline read-only connection fields remain visible, while the connection test is visible, disabled, and explains that administrator access is required.
• GIVEN connection and workload values on one Kubernetes profile page are both dirty, WHEN an administrator saves, THEN the page requests one active-session confirmation, persists each resource in dependency order, and leaves only a failed resource dirty if one API call fails.
• GIVEN a configured Kubernetes executor has at least one profile, WHEN the user selects the executor group or opens its legacy connection URL, THEN settings resolve to a profile root rather than mounting a separate connection editor.
• GIVEN a Kubernetes profile has short or long PodTemplate YAML, WHEN the value loads, is edited, or is reset, THEN the field shrinks or grows to the content, the page remains the only vertical scroll owner, and long unwrapped lines do not create document-level horizontal overflow.
• GIVEN a task uses a running Kubernetes session, WHEN its executor disclosure opens, THEN the Pod-shaped Kubernetes icon, exact Pod name, running state, restart count, and workspace mode are shown, and compact Refresh plus Profile settings icons are available in the summary header instead of a generic ready badge and empty-resource message.
• GIVEN that disclosure belongs to a Kubernetes profile, WHEN the user selects its settings icon, THEN navigation opens the exact profile root identified by executor_profile_id, not the executor-level connection route.
• GIVEN a loaded Kubernetes task disclosure, WHEN the user selects Refresh, THEN the control immediately reports an in-progress state, waits for the exact task/session status read, and then publishes the refreshed Pod facts without clearing the last known values during the request.
• GIVEN background polling has already started that exact status read, WHEN the user selects Refresh, THEN the visible refresh state remains active until the shared request settles and no duplicate read is issued.
• GIVEN a sidebar or Kanban executor hover previously observed an Unauthorized status from launch-time credentials, WHEN current kubeconfig credentials can inspect the exact recorded Pod and the indicator mounts or its disclosure opens again, THEN the status is re-read, the error clears, and the Pod icon uses the current healthy tone.
• Compact indicator acceptance criteria are maintained in REQ-EXECUTORS-TASK-STATUS-001.
• GIVEN the recorded Kubernetes Pod is pending, failed, missing, or cannot be inspected, WHEN the task disclosure refreshes, THEN it shows the sanitized live state or unavailable/error state and never reports ready solely from the task-environment row.
• GIVEN the Kubernetes task is opened with a coarse pointer, WHEN the user taps the named executor button, THEN a Drawer exposes the same Pod status and controls as desktop without depending on hover.

Out of scope

• Kubernetes operators, CRDs, Helm packaging, Deployments, Jobs, multi-Pod sessions, autoscaling, and pooled/multi-cluster scheduling.
• Cross-namespace profiles inside one executor.
• Windows Pods and automatic node-architecture discovery.
• Sharing one Pod or one managed PVC among concurrent sessions.
• Live mutation of running Pods after profile edits.
• Raw kubeconfig storage in executor JSON or the current user-scoped secret store.
• Exposing agentctl through a Service, Ingress, NodePort, or non-loopback listener.
• Direct Pod restart/delete controls in the task disclosure. Existing Stop/Resume and terminal Archive/Delete flows remain the lifecycle authority.
• Merging executor connection settings and profile settings into one database entity or API transaction. They remain separate resources even though one profile page composes and coordinates both settings surfaces.
• Profile-scoped active-session filtering. The status section deliberately shows every sanitized active session using the shared executor.
• Replacing the raw PodTemplate field with a syntax-highlighting YAML editor.
• Persisting live Pod status in TaskStatusSummary, adding a Kanban-specific backend batch endpoint, or changing the existing exact Kubernetes session and non-Kubernetes task.session.status wire contracts.

Implementation plan

See the Kubernetes executor implementation plan, the task runtime disclosure repair plan, and the Kubernetes executor UX repair plan. The follow-up status and settings hierarchy repair is tracked in the Kubernetes executor status and settings repair plan. The eager task-icon status and disclosure repair is tracked in the Kubernetes executor task status preview repair plan.