Co-authored-by: Kandev Agent <agent@kandev.dev>
16 KiB
Task Session Resume
How sessions are recovered and resumed after a backend restart or agent failure.
Overview
When the backend restarts, running agent processes are lost but all session state is persisted in SQLite (TaskSession, ExecutorRunning). The system uses a lazy recovery pattern: sessions are not immediately restarted. Instead, when a user navigates to a session, the frontend detects NeedsResume and triggers the resume flow, which relaunches agentctl + the agent process.
Two resume strategies exist depending on agent capabilities:
| Strategy | When used | What happens |
|---|---|---|
Native resume (ACP session/resume or session/load) |
Agent has NativeSessionResume: true and a stored resume token |
Agent restores its own conversation context via ACP protocol |
| History injection (fresh start) | Agent has HistoryContextInjection: true, no resume token |
Agent starts fresh; conversation history is injected into the first user prompt |
Agents with neither flag simply boot idle with no context restoration.
Resume Flow
Frontend Backend (Orchestrator) Lifecycle Manager
| | |
| task.session.status | |
|----------------------------->| |
| { needs_resume: true } | |
|<-----------------------------| |
| | |
| task.session.resume | |
|----------------------------->| ResumeTaskSession() |
| | executor.ResumeSession() |
| |------> buildResumeRequest() |
| |------> LaunchAgent() (agentctl) |
| |------> persistResumeState() |
| |------> startAgentProcessOnResume()|
| | | |
| | | (async goroutine) |
| | |----------------------->| StartAgentProcess()
| | | | configureAndStartAgent()
| | | | initializeACPSession()
| | | | dispatchInitialPrompt()
| | | | -> (see three-way switch)
| | | |
| | | callback: restore |
| | | WAITING_FOR_INPUT |
|<--- WS events (state) ------|----------|------------------------|
Entry Points
1. Frontend auto-resume (primary)
hooks/domains/session/use-session-resumption.ts
When a user opens a session page, the useSessionResumption hook:
- Sends
task.session.statusto backend - If
needs_resume && is_resumable-> sendstask.session.resume - Uses
hasAttemptedResumeref to prevent duplicate resume requests
2. GetTaskSessionStatus (status evaluation)
internal/orchestrator/task_operations.go ~line 629
Backend method backing task.session.status. Evaluates resume eligibility through 4 cases:
| Case | Condition | NeedsResume | Reason |
|---|---|---|---|
| 1 | Agent is running in memory | false | Already running |
| 2 | Agent not running but was recently seen | false | Stale execution |
| 3 | Resume token exists + active state | true | agent_not_running |
| 4 | No token but ExecutorRunning exists + active state |
true | agent_not_running_fresh_start |
Active session states: WAITING_FOR_INPUT, STARTING, RUNNING.
Terminal states (block resume): FAILED, COMPLETED, CANCELLED.
3. ensureSessionRunning (lazy resume on prompt)
internal/orchestrator/task_operations.go ~line 548
Fallback path: when PromptTask is called but no in-memory execution exists (post-restart), it triggers ResumeSession() and polls until WAITING_FOR_INPUT is reached (500ms intervals, 90s timeout).
4. handlePromptWithResume (prompt retry)
internal/task/handlers/message_handlers.go ~line 393
If a prompt fails with ErrExecutionNotFound, the handler calls ResumeTaskSession(), waits for ready, and retries the prompt.
Resume Execution (Backend)
ResumeTaskSession
internal/orchestrator/task_operations.go ~line 402
- Load session +
ExecutorRunningfrom DB - Validate worktree paths exist on disk
- Reject terminal states
- Call
executor.ResumeSession(ctx, session, startAgent=true) - On failure: mark session + task
FAILED
Executor.ResumeSession
internal/orchestrator/executor/executor_resume.go ~line 133
validateAndLockResume()-- acquire per-session mutex, verify no existing executionbuildResumeRequest()-- buildLaunchAgentRequestfrom stored configLaunchAgent()-- create agentctl instance (runtime-level)persistResumeState()-- update session state toSTARTINGin DBstartAgentProcessOnResume()-- async goroutine to start agent subprocess
buildResumeRequest (prompt decision)
internal/orchestrator/executor/executor_resume.go ~line 270
This is where the system decides whether to auto-prompt the agent:
// Case A: Native resume -- has resume token
if running.ResumeToken != "" && startAgent {
req.ACPSessionID = running.ResumeToken // restores the saved ACP session
req.TaskDescription = "" // don't auto-prompt
}
// Case B: Fresh-start resume -- no token, was WAITING_FOR_INPUT
else if startAgent && session.State == WAITING_FOR_INPUT {
req.TaskDescription = "" // don't auto-prompt; boot idle
}
// Case C: Otherwise -- TaskDescription stays populated (initial launch)
Clearing TaskDescription is critical: it controls whether dispatchInitialPrompt sends an automatic prompt or lets the agent boot idle.
startAgentProcessOnResume
internal/orchestrator/executor/executor_resume.go ~line 488
Runs StartAgentProcess in a background goroutine. On success, the callback restores the session to WAITING_FOR_INPUT and the task to REVIEW (if the session was previously in WAITING_FOR_INPUT). This uses a captured previousState variable because persistResumeState mutates the session state to STARTING before the goroutine runs.
Agent Process Initialization
StartAgentProcess
internal/agent/lifecycle/manager_startup.go ~line 22
- Wait for agentctl HTTP server ready (60s timeout)
- Configure agent (command, env, working directory)
- Start agent subprocess via agentctl
- Initialize ACP session (
session/new,session/resume, orsession/load) - Call
dispatchInitialPrompt()
ACP session restoration and creation
internal/agent/lifecycle/session.go ~line 118
If agent has NativeSessionResume: true AND existingSessionID is non-empty:
-> Agent advertises resume: session/resume without history replay
-> Method not found, load advertised, context active: session/load of the same session
-> Agent does not advertise resume: session/load
-> Only explicitly unsupported or unknown-session errors may fall back to session/new
-> Transport, authentication, cancellation, and inconclusive errors preserve the saved identity
Otherwise:
-> session/new
Replay-free resume is selected by the agent's advertised capability. Both restore
responses retain typed configuration and legacy model state. Kandev already
stores the displayed conversation, so an agent can restore its provider context
without transferring that history again. Traces distinguish the actual
acp.session.resume and acp.session.load requests beneath acp.session.restore.
dispatchInitialPrompt (three-way switch)
internal/agent/lifecycle/session.go ~line 338
| Condition | Behavior |
|---|---|
taskDescription is non-empty |
Send it as the initial prompt (standard launch) |
taskDescription empty + shouldInjectResumeContext() returns true |
Set needsResumeContext = true, mark agent ready (idle). History injected on first user prompt. |
| Neither | Mark agent ready (idle, no context) |
History Injection
For agents that cannot natively restore their session, conversation history can be injected into the first user prompt after resume. This is opt-in via the HistoryContextInjection flag on SessionConfig.
Recording
internal/agent/lifecycle/session_history.go
SessionHistoryManager stores history as JSONL files at ~/.kandev/sessions/{sessionID}.jsonl.
Entry types: user_message, agent_message, tool_call, tool_result.
Recording happens in:
SendPrompt()-- records user messages (session.go~line 510)handleAgentEvent()-- records agent messages and tool calls (manager_events.go)
Injection
When shouldInjectResumeContext() returns true (checks HistoryContextInjection flag + history file exists), the system defers injection:
dispatchInitialPrompt()setsexecution.needsResumeContext = true- Agent boots idle, session goes to
WAITING_FOR_INPUT - User sends a message
SendPrompt()callsbuildEffectivePrompt()(session.go~line 382)buildEffectivePrompt()detectsneedsResumeContext = true, callsGenerateResumeContext()- Sets
resumeContextInjected = true(one-time injection)
Resume context format
GenerateResumeContext() (session_history.go ~line 207) produces:
RESUME CONTEXT FOR CONTINUING TASK
=== EXECUTION HISTORY ===
[USER]: <message>
[ASSISTANT]: <message>
[TOOL CALL: tool_name]
[TOOL RESULT: tool_name] <result>
=== CURRENT REQUEST ===
<user's actual prompt>
=== INSTRUCTIONS ===
You are continuing work on the above task...
Do not repeat work that was already completed.
Content is truncated: user/agent messages to 2000 chars, tool results to 500 chars.
Relevant flags on AgentExecution
internal/agent/lifecycle/types.go ~line 67
| Field | Purpose |
|---|---|
historyEnabled |
Gates both recording and injection. Set from SessionConfig.HistoryContextInjection. |
needsResumeContext |
Set true when history exists and should be injected on next prompt. |
resumeContextInjected |
Set true after context has been injected (prevents double injection). |
Backend Restart Recovery
Runtime-level recovery (RecoverInstances)
internal/agent/lifecycle/manager_lifecycle.go ~line 21
On Manager.Start(), each runtime recovers what it can:
| Runtime | Recovery behavior |
|---|---|
| Docker | Lists containers with kandev.managed=true, recovers running ones, reconnects agentctl |
| Standalone | Returns nil -- processes are transient, restarted on resume |
| Sprites | Returns nil -- reconnected via metadata on resume |
| Remote Docker | Returns nil |
Lazy recovery pattern
After a backend restart:
- Docker containers may be recovered immediately (agentctl still running)
- Standalone agent processes are NOT recovered at runtime level
- All session state persists in SQLite (
ExecutorRunningretains worktree paths, resume tokens) - When frontend connects and opens a session,
GetTaskSessionStatus()returnsNeedsResume: true - Frontend auto-triggers
task.session.resume ResumeSession()creates a new agentctl instance and restarts the agent process- Depending on agent capabilities: native resume via token, or fresh start with history injection
Session State Transitions
Normal resume (success)
Before restart: Session = WAITING_FOR_INPUT Task = REVIEW
Resume starts: Session -> STARTING (persistResumeState)
Agent ready: Session -> WAITING_FOR_INPUT (startAgentProcessOnResume callback)
Task -> REVIEW
User sends msg: Session -> RUNNING Task -> IN_PROGRESS
Agent completes: Session -> WAITING_FOR_INPUT Task -> REVIEW
Resume failure (token invalid)
Resume attempted: Session -> STARTING
Token rejected: Session -> WAITING_FOR_INPUT (not FAILED)
Task -> REVIEW
ResumeToken -> cleared
Status message shown to user
The session is NOT marked as failed -- the user can send a new message to start a fresh session.
Agent Resume Capabilities
| Agent | NativeSessionResume | HistoryContextInjection | Notes |
|---|---|---|---|
| Claude Code | - | - | Uses --resume CLI flag with stored resume token |
| Codex | yes | - | Prefers advertised ACP session/resume without history replay; falls back to session/load when resume is unsupported |
| Copilot | yes | - | Prefers advertised ACP session/resume; otherwise uses session/load |
| Auggie | yes | - | Prefers advertised ACP session/resume; otherwise uses session/load (v0.18.1+) |
| Gemini | - | - | Boots idle, no context restoration |
| Amp | - | - | Uses ForkSessionCmd / ContinueSessionCmd |
| OpenCode | yes | - | Prefers advertised ACP session/resume; otherwise uses session/load |
Agent configs are defined in internal/agent/agents/.
ExecutorRunning Model
internal/task/models/models.go ~line 383
Persists runtime state for a session across backend restarts:
| Field | Purpose |
|---|---|
ResumeToken |
ACP session ID (used for session/resume, session/load, or --resume CLI flag) |
LastMessageUUID |
For --resume-session-at flag |
ContainerID |
Docker container ID (for recovery) |
AgentctlURL / AgentctlPort |
agentctl connection info |
WorktreeID / WorktreePath / WorktreeBranch |
Git worktree info (validated on resume) |
AgentExecutionID |
Links to in-memory AgentExecution |
Status |
Current executor status string |
The resume token is stored/updated in storeResumeToken() (internal/orchestrator/event_handlers.go ~line 45), called from three event handlers:
handleACPSessionCreated-- on initial session creationhandleSessionStatusEvent-- on session status updateshandleCompleteStreamEvent-- on stream completion
The token is always stored regardless of NativeSessionResume. Agents with native resume use it for ACP session/resume or session/load; others (e.g., Claude Code) use it for their --resume CLI flag.
Key Files
| File | Contains |
|---|---|
internal/orchestrator/task_operations.go |
GetTaskSessionStatus, ResumeTaskSession, ensureSessionRunning |
internal/orchestrator/executor/executor_resume.go |
ResumeSession, buildResumeRequest, persistResumeState, startAgentProcessOnResume |
internal/agent/lifecycle/session.go |
dispatchInitialPrompt, shouldInjectResumeContext, buildEffectivePrompt, SendPrompt |
internal/agent/lifecycle/session_history.go |
SessionHistoryManager, GenerateResumeContext, HasHistory |
internal/agent/lifecycle/manager_startup.go |
StartAgentProcess, initializeAgentSession |
internal/agent/lifecycle/manager_lifecycle.go |
RecoverInstances (runtime-level recovery) |
internal/agent/lifecycle/types.go |
AgentExecution fields (historyEnabled, needsResumeContext) |
internal/task/models/models.go |
TaskSession, ExecutorRunning, session state constants |
internal/orchestrator/event_handlers.go |
storeResumeToken |
hooks/domains/session/use-session-resumption.ts |
Frontend resume trigger |