Files
kandev/docs/specs/tasks/system-design/runtime-cleanup.md
T
Clément Fiere 01b0dedd6c fix: harden task archive lifecycle recovery (#3651)
Co-authored-by: Carlos Florencio <carlosmflorencio@gmail.com>
2026-09-14 16:34:25 +01:00

549 lines
32 KiB
Markdown

---
status: draft
system: tasks
requirements:
- REQ-TASKS-RUNTIME-CLEANUP-001
created: 2026-06-22
updated: 2026-09-09
owners:
- cfl
---
# Task Runtime Cleanup System Design
## Purpose and boundaries
This design defines task runtime cleanup for REQ-TASKS-RUNTIME-CLEANUP-001.
The active implementation authority is split into
[Archive Cleanup Evidence](archive-cleanup-evidence.md),
[Runtime Startup Registry](runtime-startup-registry.md), and
[Workspace Deletion Table Registry](workspace-deletion-table-registry.md).
This migrated record remains the behavioral requirement source; those records
own exact evidence, startup symbols, and table names.
## Requirement mapping
| Requirement | Design source |
| --- | --- |
| REQ-TASKS-RUNTIME-CLEANUP-001 | Migrated legacy design detail below |
## Migrated design source
Decision: [ADR-2026-08-08-task-owned-worktree-lifetime](../../../decisions/2026-08-08-task-owned-worktree-lifetime.md)
## Why
Operators archive and delete completed tasks to keep the board usable and the
workspace tidy. Those actions must also release the task's runtime resources;
otherwise completed work leaves hidden ACP processes, host utility processes,
worktrees, or executor rows behind and the machine slowly runs out of memory.
## What
- Archive and delete durably inventory every runtime still associated with the
task before removing live task, worktree, or runtime-tracking rows. Physical
stop is attempted by the cleanup worker outside the deletion transaction.
- Cleanup ownership is based on `executors_running`, not only on active
`task_sessions` state. A runtime row for a completed, cancelled, archived, or
otherwise terminal session is still a cleanup target until the runtime stop
has been attempted.
- Before deleting live runtime/session/environment/repository rows, cleanup
persists an immutable `task_resource_cleanup_snapshots` record containing the
exact handles and ownership proof. That retained record remains authoritative
until stop/cleanup succeeds or a retryable failure is preserved.
- Worktree and task-environment cleanup is ownership-aware. A task cleanup may
destroy only resources owned by that task's `TaskEnvironment`; borrowed or
inherited worktrees remain owned by the source task.
- A `TaskEnvironment` referenced by another active task session is shared for the
duration of that session. Cleanup can stop the current task's runtime rows, but
must defer destructive worktree/container/sandbox teardown until no other
active task session references the environment.
- A task owns its materialized workspace independently of its sessions. Deleting
any session, including the task's last session, removes only that session and
its workspace references. It does not remove task workspace directories, Git
worktree registrations, or branches.
- A task with zero sessions retains its materialized workspace so a later session
can reuse the same files and Git state.
- Physical workspace cleanup is initiated only by task lifecycle operations:
archive, delete, cascade archive/delete, workspace delete, quick-chat expiry,
or explicit task-environment reset. Session deletion is not a cleanup trigger.
- Agent shutdown escalates to the whole process group when its timeout expires
and waits for descendants; neither agentctl nor backend shutdown reports
completion while an owned process remains under PID 1.
- Standalone agentctl ignores terminal foreground interrupts so the backend owns
Ctrl+C sequencing. Launchers gracefully terminate the backend before force
kill, and backend shutdown includes agentctl's cleanup window.
- Startup reconciliation classifies stale runtime ownership and liveness.
Confirmed-dead local rows clean idempotently; alive, unknown, remote, or
generic failures remain durable. Expected fail-closed cases are aggregated in
a bounded structured summary; unexpected failures retain individual warnings.
- Cleanup is idempotent only for classifiable absence. Persisted stop handles and
runtime-aware liveness may complete confirmed-dead local rows; alive, unknown,
remote, and lookup failures remain retryable.
- Archive/delete/cascade/workspace-delete/quick-chat expiry persist cleanup intent
and snapshot before task mutation; a durable worker, never a detached goroutine,
owns execution. A contended advisory stop guard does not block requested stop.
- `bounded_terminal` retries eight times with 1m, 5m, 15m, 1h, 3h, 6h, and 12h
delays. Cascade-critical archive/unarchive retains attempt-eight diagnostics and
continues every 12h under `cascade_retry`.
- Archive cleanup revalidates archived state before destruction. Unarchive
cancels pending archive cleanup and preserves historical worktree/branch data.
- Attach-only resume requires the same executor owner and a live durable worktree
row. Otherwise normal preparation recreates the directory/branch using the
historical worktree ID when available, then reactivates or upserts the matching
environment/repository/branch row.
## Archive cascade coordination
[Durable Archive Cascades](durable-archive-cascades.md) owns operation
reservation, immutable membership, workspace authorization, partial-mutation
recovery, unarchive, and parent-mutation admission. The architecture choice is
recorded in [Persist task cascade mutation progress](../../../decisions/2026-09-05-durable-task-cascade-mutations.md).
This cleanup design owns the caller-cancellation boundary after preparation.
The request computes one absolute deadline: the earlier caller deadline or entry
plus `TaskArchiveTimeout` (two minutes). Authorization, reservation, and cleanup
preparation remain caller-cancellable. After preparation, request-owned runtime
stop, mutation, finalization, publication, vacancy, activation, rollback, and
compensation use cancellation-independent contexts capped by
`min(original_deadline, now+step_budget)`. No “fresh” context extends it.
When the original deadline has expired, synchronous recovery stops after
persisting retry direction/error and waking the mandatory runtime; the response
returns the durable operation identity. A later worker claim has its own bounded
attempt deadline and is not part of the expired request attempt. Exact nested
budgets and expiry tests are in
[Archive Cascade Boundary Contracts](archive-cascade-boundary-contracts.md#absolute-request-deadline).
Task/group jobs use the full execution-contract state machine.
Cleaner/restorer receives versioned database-scoped `CleanupFence` keys, exact
snapshot/lifecycle marker identity, and provider proof. Claim commits before
guard wait; no SQL transaction waits. Short marker/completion transactions
bracket I/O. Unarchive advances lifecycle generation before joining cleanup;
restore uses identical keys under that newer fence.
## Archive cleanup disposition
Direct and cascade archive share one disposition: stop task runtimes, remove the
physical worktree, mark its repository row deleted, and retain the owning
`task_environments` plus every `task_environment_repos` identity/path/branch/slug
and local branch ref. Duplicate snapshot references cannot delete a ref retained
by an earlier pass. Unarchive reuses that environment and branch; absent live
repository rows use normal preparation. Delete may remove owner rows only after
capturing the cleanup snapshot.
## Data Model
Preparation with absent resources follows
[Cleanup preparation](runtime-cleanup-preparation.md).
### `executors_running`
`executors_running` remains the durable runtime ownership table and the source of
cleanup handles for task runtime teardown.
- `session_id` identifies the task session that originally launched the runtime.
- `task_id` identifies the owning task and is the primary lookup key for archive
and delete cleanup.
- `agent_execution_id` is the preferred stop handle for in-memory runtime
shutdown through the lifecycle manager.
- `runtime`, `container_id`, `agentctl_url`, `agentctl_port`, `pid`, and
`metadata` provide fallback handles for runtime-specific cleanup and
diagnostics.
- `status`, `error_message`, and timestamps remain available for cleanup
diagnostics when stop fails and the row must remain retryable.
Rows may temporarily reference archived tasks or terminal sessions while cleanup
is in progress. Rows must not reference missing task/session state indefinitely.
### `task_sessions`
`task_sessions.state` remains the user-facing session state. Terminal states do
not imply runtime resources have been released. Cleanup code must not use terminal
session state as a reason to skip runtime teardown when an `executors_running`
row exists.
### `task_environments` and `task_environment_repos`
`task_environments.task_id` is the canonical workspace owner.
`task_environment_repos` records the ordered repositories in that environment
and is the single source of physical worktree identity, path, branch, status, and
lifecycle timestamps. Task cleanup queries repository rows through the owning
environment's `task_id`; it never needs a session row.
Sessions reference shared workspaces through `task_environment_id`. A
transactional SQLite/PostgreSQL migration backfills `task_session_worktrees`
into `task_environment_repos`, removes redundant columns, and leaves no dual
path. It fails closed unless every legacy identity and path has one owner.
Under the migration lock it verifies shadow inventory, ownership, constraints,
row counts, and final schema before commit; failure rolls back. SQLite requires
the pre-upgrade snapshot, while PostgreSQL uses transactional DDL and its
advisory lock. Migration performs no filesystem/Git work; fresh databases use
the final schema.
### `task_resource_cleanup_jobs`
`task_resource_cleanup_jobs` is durable task-lifecycle intent. It has no foreign
key to live task, session, runtime, environment, group, or physical rows, so
delete cannot remove its immutable resource snapshot. It stores operation,
workspace, task, resource-key set, snapshot hash/payload, target/work direction,
lifecycle generation, state/retry/error, claim generation/owner/lease/heartbeat,
unknown-resolution evidence, and timestamps.
Task cleanup uses the exact task/group transition table in
[Archive Cascade Execution Contracts](archive-cascade-execution-contracts.md#task-and-group-cleanup-jobs).
Cascade archive/unarchive uses unbounded `cascade_retry`: after the normal seven
delays, failures retain an exhausted diagnostic and retry every 12 hours.
Non-restorable delete/shutdown may use `bounded_terminal` and enter retained
`failed` on attempt eight. Ambiguous I/O always enters `unknown`.
A prepared job is also the durable creation barrier. Every production writer to
session, runtime, environment, and physical-handle tables is registered as
admitted or column-scoped exempt in the aggregate writer inventory. Admitted
writes serialize against the task/operation barrier; writer-first state enters
the snapshot and reservation-first returns typed conflict. The barrier
transaction holds no filesystem, target-path, or Git lock.
## API Surface
No new action is required for the base contract. Dirty deletion admission is in
[Dirty Worktree Task Deletion](dirty-worktree-deletion.md).
`session.delete` keeps its existing request and response contract. Success means
the session row is gone. It does not mean the task workspace was cleaned, and it
never enqueues task resource cleanup.
Internal contracts:
```go
type ExecutorRunningRepository interface {
ListExecutorsRunningByTaskID(ctx context.Context, taskID string) ([]*models.ExecutorRunning, error)
ListExecutorsRunning(ctx context.Context) ([]*models.ExecutorRunning, error)
}
```
`TaskExecutionStopper.StopExecution(ctx, executionID, reason, force)` remains the
primary stop operation when `agent_execution_id` is available. Fallback cleanup is
runtime-specific and must be bounded by context.
`TaskExecutionStopper.RegisterExecutionStopOwner(sessionID, executionID, force)`
must not wait for the session guard; it may skip a contended claim and never
replaces `StopExecution`.
## State Machine
Runtime cleanup for a task follows this lifecycle:
- `tracked`: an `executors_running` row exists for the task.
- `stop_requested`: archive, delete, explicit stop, terminal-agent cleanup, or
startup reconciliation selected the row.
- `snapshot_handed_off`: task/cascade/workspace deletion committed an immutable
resource snapshot; the live row may be removed, but the retained handle owns
cleanup.
- `stopped`: the runtime instance and subprocess group exited or were absent.
- `tracking_removed`: the live runtime row is deleted after stop, or after the
snapshot handoff that preserves its authoritative handle.
Allowed transitions:
- `tracked` -> `stop_requested` by archive/delete/session stop/reconciliation.
- `stop_requested` -> `snapshot_handed_off` for task, cascade, or workspace
deletion after its immutable snapshot commits, or -> `stopped` when shutdown
succeeds/absence is proved.
- `snapshot_handed_off` -> `tracking_removed` after the live row is deleted.
- `stopped` -> `tracking_removed` after cleanup no longer needs the live handle.
- `stop_requested` -> `retryable_failure` on timeout or uncertain state.
- `retryable_failure` -> `stop_requested` on the next cleanup attempt.
The deleting transaction commits the snapshot, cleanup claim, and handoff
generation before removing the live runtime row. The cleanup worker owns the
stop from that claim; lease expiry permits a higher-generation takeover, and a
crash after commit cannot leave an unowned running runtime.
The durable task cleanup job follows the authoritative
[task/group table](archive-cascade-execution-contracts.md#task-and-group-cleanup-jobs).
It stores desired `target_direction` separately from the claimed
`work_direction`.
Unarchive sets target restore and advances lifecycle generation before joining a
running cleanup claim. Prepared/pending/unclaimed retry jobs cancel directly.
Running or unknown cleanup becomes `unknown` under the new fence; reconciliation
acquires the same physical guard and proves either no cleanup (`cancelled`) or
completed cleanup (`restore_pending`). It never labels possibly completed I/O
cancelled from lease state alone.
Restore claims move `restore_pending|restore_retry_wait` through `restoring` to
`restored`; ambiguous/expired work enters `unknown` until proof selects restored
or safe retry. Task restore validates retained environment/repository/branch
metadata and marks it safely materializable; normal admitted session preparation
recreates an absent worktree later. Archive requires every required nonmissing
cascade job `succeeded`; unarchive requires each `cancelled` or `restored`.
Concurrently deleted members instead retain `delete_owned` cleanup and are
reported missing rather than exposed.
## Failure Modes
- If querying `executors_running` for a task fails, archive/delete still updates
the task only if existing product behavior requires it, but destructive runtime
cleanup must fail closed: do not remove runtime rows or worktrees based on an
empty or partial inventory.
- A contended owner claim is skipped; explicit stop continues and durable cleanup
preserves retryable work.
- If stopping a runtime execution times out, the process manager escalates to a
process-group kill and waits for confirmation. If confirmation still fails, the
runtime row remains retryable.
- If a runtime row points at a missing in-memory execution, cleanup attempts the
runtime-specific persisted handle when available. If no handle can be used, the
row is preserved with a warning instead of being silently dropped.
- Keep `models.ErrExecutionRotated` (newer won); warn on other errors.
- If a stop operation reports the execution or session is not found and the owned
row is a confirmed-dead local runtime, cleanup records the stop as successful,
prunes or repairs the row under the resume-safety invariant, proceeds with any
now-safe shared-environment/worktree teardown, and does not count the row as a
failed stop. The durable cleanup job therefore does not re-enter `retry_wait`
solely because that owned runtime was confirmed absent.
- If a stop operation reports not found but the owned row is alive, or its
liveness is Unknown (remote/containerized/no local handle), cleanup preserves
the row and treats the outcome as retryable rather than pruning it.
- If a session or task lookup during stop fails with an error that is not a typed
not-found sentinel, cleanup treats it as a retryable failure and preserves the
row; it does not reinterpret the error as an absent runtime.
- If worktree or task-environment cleanup fails after runtime shutdown is
confirmed, the runtime tracking row can still be removed because it no longer
identifies a live process. The resource cleanup error remains retryable.
- An absent captured database row is an idempotent metadata-deletion success; it
says nothing about a physical path. An absent physical worktree/environment
path succeeds only when the authenticated external ownership marker and locked
Git registration prove that exact snapshot was already removed. Generic
filesystem or Git not-found is `unknown`, preserves evidence, and retries.
- Bounded-policy delete/shutdown jobs may enter retained terminal `failed` on
claim eight, except DeleteTask Git registration/branch cleanup, which always
uses unbounded generation-fenced `cascade_retry`. Other cascade jobs retain
`last_error`, set `exhausted`, and schedule the next claim in 12 hours.
- If cleanup cannot prove that a session worktree belongs to the task being
cleaned, destructive worktree deletion fails closed and skips that worktree.
Stale Git state is removed only with pinned path, branch, and commit ownership.
- If an agentctl process exits unexpectedly, its owned agent subprocess group is
killed before agentctl shutdown completes.
- If the user sends Ctrl+C to a standalone Kandev process tree, agentctl does
not receive the terminal interrupt directly; it is stopped through backend
lifecycle shutdown, parent liveness, or an explicit backend signal.
- If the user sends Ctrl+C while running through `make start-debug`, the launcher
forwards a graceful stop to the backend and waits before escalating, rather
than immediately killing the backend process group.
- If startup reconciliation finds rows for archived tasks, deleted tasks, missing
sessions, or terminal sessions with no live runtime, it removes only rows that
are positively confirmed safe to remove.
- If startup reconciliation receives a typed runtime-not-found result for a local
row with no persisted process liveness handle, it classifies liveness as
Unknown and preserves the row. It summarizes that expected fail-closed outcome
instead of emitting one ambiguous warning per legacy row.
- Startup diagnostics identify the runtime, session state, liveness class,
presence of a local PID, stop-error class, disposition, and aggregate count.
They never include resume tokens, credentials, or provider payloads.
- If cleanup intent or its resource snapshot cannot be persisted, the lifecycle
mutation fails before destructive cleanup begins; Kandev does not rely on an
unrecorded background goroutine.
- If an archived task is unarchived before cleanup completes, the worker cancels
remaining archive cleanup. Already completed resource removal remains valid and
the unarchive branch-recovery flow recreates the environment when possible.
- If an unarchived worktree environment has no live physical repository row,
resume does not enter attach-only preparation. It enters normal worktree
preparation, where unavailable branch recovery remains a typed worktree
recovery failure rather than `ErrReuseWorktreeUnavailable`.
- If deleting a session fails, no physical workspace operation has been attempted;
the task-owned worktree record remains authoritative regardless of whether the
session mutation committed.
- If a session or worktree creation races task archive/delete preparation, the
task lifecycle barrier wins before cleanup inventory is finalized. The creation
is rejected or its uncommitted materialization is compensated, and the cleanup
snapshot cannot omit a newly owned worktree.
- If task-owned worktree backfill encounters conflicting ownership or path data,
startup fails closed with a diagnostic instead of choosing a row that could
authorize deletion of another task's workspace.
## Persistence Guarantees
- Runtime cleanup intent survives backend restarts because live
`executors_running` rows remain until stop is attempted; after live-row
deletion, `task_resource_cleanup_snapshots` retains the exact handle,
ownership, and proof envelope until cleanup succeeds or retryable failure is
recorded.
- Runtime stop is represented in the retained snapshot before live worktree or
task-environment rows are removed; the cleanup worker attempts the stop and
later physical teardown from that snapshot.
- Startup reconciliation reloads retained snapshots and stale runtime rows.
A typed-not-found, confirmed-dead local row is removed only after its
snapshot records the result; unknown liveness remains durable.
- Pending and retryable task cleanup jobs survive restart and resume independently
of whether optional scheduled storage maintenance is enabled.
- Terminal `failed` bounded-policy jobs survive restart for diagnosis and do not
resume automatically; cascade-policy jobs never enter that state.
- Cleanup snapshots needed after task deletion survive without foreign-keyed task,
session, environment, or worktree rows.
- A `task_environment_repos` row, its directory, Git registration, branch, and
uncommitted files survive deletion of every session. Task lifecycle deletion
snapshots those identities before removing live rows; the retained snapshot
remains discoverable for cleanup and proof.
- Storage inventory protects paths from task environment repository rows, not
only paths referenced by live sessions, so a zero-session task workspace is
not classified as orphaned.
- Historical worktree rows for archived tasks remain available to unarchive branch
recovery even after their on-disk directories are removed.
- Archive cleanup preserves the task environment owner and the local branch ref.
Cleanup retries and duplicate teardown passes keep the same disposition.
- Orphaned OS processes without any durable `executors_running` row are outside
normal cleanup guarantees; they may be handled by an explicit operator recovery
tool, but automatic task cleanup must not rely on process-name scanning.
## Scenarios
- **GIVEN** a task with one stopped session, a registered worktree, and
uncommitted files, **WHEN** the session is deleted, **THEN** the task remains
with zero sessions and the directory, Git registration, branch, and files are
unchanged.
- **GIVEN** a task whose last session was deleted, **WHEN** the backend restarts
and a new session is created, **THEN** the session reuses the task-owned
workspace and observes the preserved files.
- **GIVEN** two sessions referencing the same task worktree, **WHEN** either
session is deleted, **THEN** the remaining session continues using the same
directory and no filesystem or Git cleanup command runs.
- **GIVEN** a session has been deleted and the backend restarted, **WHEN** its
task is archived, **THEN** durable asynchronous task cleanup still discovers
the task-owned worktree by `task_id` and removes it unless another task holds
a protected shared reference.
- **GIVEN** a session has been deleted and the backend restarted, **WHEN** its
task is deleted, **THEN** the cleanup job snapshot retains the worktree handles
after task-row cascades and retries filesystem or Git failures after restart.
- **GIVEN** task cleanup preparation is racing a new session/worktree launch,
**WHEN** the lifecycle barrier is reserved, **THEN** no resource created after
the barrier can be omitted from the cleanup inventory or survive as an
untracked directory.
- **GIVEN** a task has a `WAITING_FOR_INPUT` session and an
`executors_running` row, **WHEN** the task is archived, **THEN** the runtime is
stopped by `agent_execution_id` before the row and worktrees are removed.
- **GIVEN** a task has a `COMPLETED` session but still has an
`executors_running` row, **WHEN** the task is deleted, **THEN** cleanup still
selects that row and attempts runtime shutdown.
- **GIVEN** runtime shutdown succeeds for every row owned by a deleted task,
**WHEN** cleanup finishes, **THEN** the task's `executors_running` rows and
worktrees are removed.
- **GIVEN** runtime shutdown times out for one row owned by an archived task,
**WHEN** cleanup finishes, **THEN** the row remains retryable with a diagnostic
error and worktree deletion does not erase the only runtime handle.
- **GIVEN** the backend restarts with an `executors_running` row for an archived
task, **WHEN** startup reconciliation runs, **THEN** it attempts cleanup for the
row instead of treating the archived task as active.
- **GIVEN** a missing, terminal, or failed session whose local runtime is already
absent and whose persisted PID is confirmed dead, **WHEN** startup
reconciliation receives a typed not-found result, **THEN** it safely removes or
repairs the row without an individual warning, and a second reconciliation has
no stale row to process.
- **GIVEN** a typed not-found result for an alive local row, a local row without a
liveness handle, or a remote row, **WHEN** startup reconciliation runs, **THEN**
it preserves the row and includes the safe disposition in a bounded startup
summary.
- **GIVEN** a stop fails with a generic error, **WHEN** startup reconciliation
runs, **THEN** it preserves the row and emits an individual structured warning
rather than reclassifying the runtime as absent.
- **GIVEN** an `executors_running` row whose task session is missing and whose
local liveness handle refers to a dead host process, **WHEN** startup
reconciliation stops it and the stop reports the execution is not found, **THEN**
the row is pruned or repaired under the resume-safety invariant instead of being
preserved indefinitely.
- **GIVEN** a resumed durable cleanup job whose only remaining runtime target is a
confirmed-dead local runtime, **WHEN** the cleanup worker stops it and the stop
reports not found, **THEN** the job records a successful stop, completes owned
resource teardown, and does not re-enter `retry_wait` because of that runtime.
- **GIVEN** an `executors_running` row that is still alive on this host, **WHEN**
a stop reports not found for it, **THEN** the row is preserved and the outcome is
treated as retryable rather than pruned.
- **GIVEN** an `executors_running` row for a remote (SSH) or containerized runtime
whose local liveness is Unknown, **WHEN** a stop reports not found, **THEN** the
row is preserved with its remote handle and resume token intact and is not
pruned on the basis of a host-local not-found.
- **GIVEN** a session lookup during a stop fails with a non-not-found store error,
**WHEN** cleanup evaluates the result, **THEN** it preserves the row and retries
rather than treating the runtime as absent.
- **GIVEN** a confirmed-dead local row that still carries a resume token, **WHEN**
a not-found stop is reclassified as successful, **THEN** the row is repaired in
place (token and worktree preserved) rather than deleted, per
`RowMustBePreserved`.
- **WHEN** compare-and-set returns `models.ErrExecutionRotated`, **THEN** keep
the newer row silently.
- **GIVEN** agentctl is stopped while an ACP child process ignores stdin EOF,
**WHEN** the stop timeout expires, **THEN** the ACP process group is killed and
no ACP child is reparented to PID 1.
- **GIVEN** an ACP wrapper process exits after spawning a native child in the
same process group, **WHEN** agentctl completes shutdown, **THEN** the
remaining process-group descendants are terminated before shutdown is reported
complete.
- **GIVEN** standalone Kandev owns an active agentctl instance, **WHEN** the user
stops Kandev with Ctrl+C, **THEN** backend shutdown supervises agentctl
cleanup and waits for the instance stop window instead of letting agentctl exit
directly from the terminal interrupt.
- **GIVEN** Kandev is running under `make start-debug`, **WHEN** the user stops it
with Ctrl+C, **THEN** the launcher gives the backend a graceful shutdown window
before any force kill so active ACP process groups are reaped.
- **GIVEN** the `executors_running` query fails during archive cleanup, **WHEN**
cleanup evaluates destructive actions, **THEN** it logs the failure and does not
delete runtime tracking rows based on incomplete information.
- **GIVEN** a child task reuses its parent's `TaskEnvironment`, **WHEN** the child
is archived or deleted, **THEN** cleanup stops the child's runtime rows without
deleting the inherited parent worktree.
- **GIVEN** a parent task owns a `TaskEnvironment` that an active child session
still references, **WHEN** parent cleanup runs, **THEN** destructive
environment and worktree teardown is deferred until the child no longer holds
the environment.
- **GIVEN** the backend exits after a task is deleted but before its worktree is
removed, **WHEN** the backend restarts, **THEN** the durable cleanup job retries
using its captured resource snapshot.
- **GIVEN** a delete cleanup snapshot references a physical worktree that is
already absent, **WHEN** the cleanup worker runs, **THEN** it succeeds only
after the authenticated external marker and locked Git registration prove
exact prior removal; generic path not-found is `unknown` and retries.
- **GIVEN** a delete cleanup snapshot requests deletion of a captured
task-environment database row that is already absent, **WHEN** the cleanup
worker runs, **THEN** the metadata step is idempotently complete and does not
retry because of that absence.
- **GIVEN** a cleanup job fails for a genuinely retryable reason, **WHEN** fewer
than eight attempts have run, **THEN** it enters `retry_wait` using the
documented backoff schedule.
- **GIVEN** a bounded-policy cleanup job fails on attempt eight, **WHEN** the
worker records it, **THEN** it enters retained terminal `failed`.
- **GIVEN** a cascade-policy job fails on attempt eight or later, **WHEN** the
worker records it, **THEN** it retains the exhausted diagnostic and schedules
another generation-fenced attempt after 12 hours.
- **GIVEN** an archive cleanup job is pending, **WHEN** the task is unarchived,
**THEN** the job is cancelled and cannot delete resources created after
unarchive.
- **GIVEN** archive cleanup removed a local worktree, **WHEN** the task is
unarchived, **THEN** its historical worktree branch metadata remains available
for local/remote recovery.
- **GIVEN** a direct or cascade archive removes a task worktree, **WHEN** the
remote branch is also absent, **THEN** the task environment, repository row,
and local branch ref remain available for normal resume preparation.
- **GIVEN** an unarchived task environment whose worktree repository row is
deleted, failed, or tombstoned, **WHEN** its session resumes, **THEN** the
executor selects normal worktree preparation and recreates or reactivates the
recoverable task-owned worktree instead of demanding attach-only reuse.
## Out of Scope
- A general-purpose OS process sweeper that kills every process named
`codex-acp`, `claude-acp`, or `opencode`.
- UI changes for showing runtime cleanup failures.
- Per-session warnings about uncommitted or unpushed workspace state; ordinary
session deletion preserves that state. Destructive warnings remain attached
to task archive/delete/reset surfaces.
- A user-facing action to replay a terminal failed cleanup job.
- New user-facing archive/delete controls.
- Changing the task/session state model beyond the cleanup guarantees described
here.
## Implementation plan
- [Backend failure containment](../../../plans/backend-failure-containment/plan.md)
- [Worktree resume after unarchive](../../../plans/worktree-resume-after-unarchive/plan.md)
- [Archive resume identity](../../../plans/archive-resume-identity/plan.md)