kandev/docs/specs/tasks/runtime-cleanup.md

---
status: draft
created: 2026-06-22
owner: cfl
---

# Task Runtime Cleanup

## Why

Operators archive and delete completed tasks to keep the board usable and the
workspace tidy. Those actions must also release the task's runtime resources;
otherwise completed work leaves hidden ACP processes, host utility processes,
worktrees, or executor rows behind and the machine slowly runs out of memory.

## What

- Archiving a task stops every runtime execution that is still durably associated
  with that task before the task's worktrees or runtime tracking rows are removed.
- Deleting a task stops every runtime execution that is still durably associated
  with that task before task records, worktrees, or runtime tracking rows are
  removed.
- Cleanup ownership is based on `executors_running`, not only on active
  `task_sessions` state. A runtime row for a completed, cancelled, archived, or
  otherwise terminal session is still a cleanup target until the runtime stop has
  been attempted.
- A cleanup path MUST NOT delete the last durable runtime handle for a task until
  either the matching runtime execution has stopped successfully or the failure is
  preserved for retry/diagnosis.
- Agent subprocess shutdown kills the whole agent process group when graceful
  shutdown does not finish within the configured stop timeout.
- Agent subprocess shutdown does not treat the command leader exiting as
  sufficient while descendants remain alive in the same process group.
- Agentctl instance shutdown never leaves an agent subprocess group alive as a
  child of `init`/PID 1.
- Standalone agentctl is isolated from terminal foreground interrupts so the
  backend remains the owner of Ctrl+C shutdown sequencing.
- Top-level launchers, including `make start-debug` through `kandev start`, send
  the backend a graceful termination signal before any force kill so backend
  shutdown can stop agents and agentctl instances.
- Backend shutdown waits long enough for standalone agentctl's instance cleanup
  window before reporting shutdown complete.
- Backend startup reconciles stale runtime rows for archived/deleted/missing task
  state and attempts safe cleanup instead of treating the rows as live sessions.
- Cleanup is idempotent: repeating archive/delete cleanup, startup reconciliation,
  or explicit session stop does not fail because the process, task, session, or
  worktree was already removed.

## Data Model

### `executors_running`

`executors_running` remains the durable runtime ownership table and the source of
cleanup handles for task runtime teardown.

- `session_id` identifies the task session that originally launched the runtime.
- `task_id` identifies the owning task and is the primary lookup key for archive
  and delete cleanup.
- `agent_execution_id` is the preferred stop handle for in-memory runtime
  shutdown through the lifecycle manager.
- `runtime`, `container_id`, `agentctl_url`, `agentctl_port`, `pid`, and
  `metadata` provide fallback handles for runtime-specific cleanup and
  diagnostics.
- `status`, `error_message`, and timestamps remain available for cleanup
  diagnostics when stop fails and the row must remain retryable.

Rows may temporarily reference archived tasks or terminal sessions while cleanup
is in progress. Rows must not reference missing task/session state indefinitely.

### `task_sessions`

`task_sessions.state` remains the user-facing session state. Terminal states do
not imply runtime resources have been released. Cleanup code must not use terminal
session state as a reason to skip runtime teardown when an `executors_running`
row exists.

## API Surface

No new user-facing HTTP or WebSocket action is required. Existing task archive,
task delete, session stop, and backend startup behavior gain stronger cleanup
guarantees.

Internal contracts:

```go
type ExecutorRunningRepository interface {
    ListExecutorsRunningByTaskID(ctx context.Context, taskID string) ([]*models.ExecutorRunning, error)
    ListExecutorsRunning(ctx context.Context) ([]*models.ExecutorRunning, error)
}
```

`TaskExecutionStopper.StopExecution(ctx, executionID, reason, force)` remains the
primary stop operation when `agent_execution_id` is available. Fallback cleanup is
runtime-specific and must be bounded by context.

## State Machine

Runtime cleanup for a task follows this lifecycle:

- `tracked`: an `executors_running` row exists for the task.
- `stop_requested`: archive, delete, explicit stop, terminal-agent cleanup, or
  startup reconciliation has selected the row for cleanup.
- `stopped`: the runtime instance and its subprocess group have exited or were
  already absent.
- `tracking_removed`: the `executors_running` row is deleted after the stop
  result is known.
- `retryable_failure`: stop could not be confirmed before the timeout. The row
  remains durable with enough context to retry and diagnose.

Allowed transitions:

- `tracked` -> `stop_requested` by archive/delete/session stop/reconciliation.
- `stop_requested` -> `stopped` when runtime shutdown succeeds or the runtime is
  confirmed absent.
- `stopped` -> `tracking_removed` after worktree/environment cleanup has been
  attempted and the runtime row is no longer needed as the durable stop handle.
- `stop_requested` -> `retryable_failure` on timeout or uncertain runtime state.
- `retryable_failure` -> `stop_requested` on the next cleanup attempt.

## Failure Modes

- If querying `executors_running` for a task fails, archive/delete still updates
  the task only if existing product behavior requires it, but destructive runtime
  cleanup must fail closed: do not remove runtime rows or worktrees based on an
  empty or partial inventory.
- If stopping a runtime execution times out, the process manager escalates to a
  process-group kill and waits for confirmation. If confirmation still fails, the
  runtime row remains retryable.
- If a runtime row points at a missing in-memory execution, cleanup attempts the
  runtime-specific persisted handle when available. If no handle can be used, the
  row is preserved with a warning instead of being silently dropped.
- If worktree or task environment cleanup fails after runtime shutdown is
  confirmed, the runtime tracking row can still be removed because it no longer
  identifies a live process. The resource cleanup error is logged and handled by
  the resource-specific retry path.
- If an agentctl process exits unexpectedly, its owned agent subprocess group is
  killed before agentctl shutdown completes.
- If the user sends Ctrl+C to a standalone Kandev process tree, agentctl does
  not receive the terminal interrupt directly; it is stopped through backend
  lifecycle shutdown, parent liveness, or an explicit backend signal.
- If the user sends Ctrl+C while running through `make start-debug`, the launcher
  forwards a graceful stop to the backend and waits before escalating, rather
  than immediately killing the backend process group.
- If startup reconciliation finds rows for archived tasks, deleted tasks, missing
  sessions, or terminal sessions with no live runtime, it removes only rows that
  are positively confirmed safe to remove.

## Persistence Guarantees

- Runtime cleanup intent survives backend restarts because `executors_running`
  rows remain durable until cleanup succeeds or a retryable failure is recorded.
- Worktrees and task environment rows are not removed before runtime stop has been
  attempted for every runtime row owned by the task.
- Startup reconciliation is allowed to recover from a previous backend crash by
  reattempting cleanup for stale runtime rows.
- Orphaned OS processes without any durable `executors_running` row are outside
  normal cleanup guarantees; they may be handled by an explicit operator recovery
  tool, but automatic task cleanup must not rely on process-name scanning.

## Scenarios

- **GIVEN** a task has a `WAITING_FOR_INPUT` session and an
  `executors_running` row, **WHEN** the task is archived, **THEN** the runtime is
  stopped by `agent_execution_id` before the row and worktrees are removed.
- **GIVEN** a task has a `COMPLETED` session but still has an
  `executors_running` row, **WHEN** the task is deleted, **THEN** cleanup still
  selects that row and attempts runtime shutdown.
- **GIVEN** runtime shutdown succeeds for every row owned by a deleted task,
  **WHEN** cleanup finishes, **THEN** the task's `executors_running` rows and
  worktrees are removed.
- **GIVEN** runtime shutdown times out for one row owned by an archived task,
  **WHEN** cleanup finishes, **THEN** the row remains retryable with a diagnostic
  error and worktree deletion does not erase the only runtime handle.
- **GIVEN** the backend restarts with an `executors_running` row for an archived
  task, **WHEN** startup reconciliation runs, **THEN** it attempts cleanup for the
  row instead of treating the archived task as active.
- **GIVEN** agentctl is stopped while an ACP child process ignores stdin EOF,
  **WHEN** the stop timeout expires, **THEN** the ACP process group is killed and
  no ACP child is reparented to PID 1.
- **GIVEN** an ACP wrapper process exits after spawning a native child in the
  same process group, **WHEN** agentctl completes shutdown, **THEN** the
  remaining process-group descendants are terminated before shutdown is reported
  complete.
- **GIVEN** standalone Kandev owns an active agentctl instance, **WHEN** the user
  stops Kandev with Ctrl+C, **THEN** backend shutdown supervises agentctl
  cleanup and waits for the instance stop window instead of letting agentctl exit
  directly from the terminal interrupt.
- **GIVEN** Kandev is running under `make start-debug`, **WHEN** the user stops it
  with Ctrl+C, **THEN** the launcher gives the backend a graceful shutdown window
  before any force kill so active ACP process groups are reaped.
- **GIVEN** the `executors_running` query fails during archive cleanup, **WHEN**
  cleanup evaluates destructive actions, **THEN** it logs the failure and does not
  delete runtime tracking rows based on incomplete information.

## Out of Scope

- A general-purpose OS process sweeper that kills every process named
  `codex-acp`, `claude-acp`, or `opencode`.
- UI changes for showing runtime cleanup failures.
- New user-facing archive/delete controls.
- Changing the task/session state model beyond the cleanup guarantees described
  here.