kandev/docs/design/dynamic-model-switching.md

Dynamic Model Switching Feature Analysis

Status: Design Document Created: 2026-01-21 Scope: Mid-session model switching within the Kandev application

Executive Summary

This document analyzes the implementation of dynamic model switching, allowing users to change the AI model mid-session without starting a new session. The feature requires UI changes to expose model selection, backend logic to detect model changes and restart agents, and careful handling of session continuity.

Current Architecture Overview

Session Lifecycle (Current Flow)

User clicks "Start Session"
    → Frontend sends orchestrator.start with agentProfileId
    → Orchestrator.StartTask → Executor.ExecuteWithProfile
    → Lifecycle Manager resolves profile → gets model from AgentProfileInfo
    → CommandBuilder.BuildCommand adds --model flag
    → Agent process started with fixed model
    → ACP session initialized
    → User sends prompts via session/prompt

Key Insight: Model is determined at session start via AgentProfileInfo.Model and baked into the agent command. There's no current mechanism to change the model mid-session.

Relevant Components

Component	Path	Role
TaskChatPanel	apps/web/components/task/task-chat-panel.tsx	Chat UI, displays current model
SessionsDropdown	apps/web/components/task/sessions-dropdown.tsx	Session switcher
Orchestrator Service	apps/backend/internal/orchestrator/service.go	PromptTask, StartTask, ResumeTaskSession
Executor	apps/backend/internal/orchestrator/executor/executor.go	Agent launch, prompt forwarding
Lifecycle Manager	apps/backend/internal/agent/lifecycle/manager.go	Agent process management
CommandBuilder	apps/backend/internal/agent/lifecycle/command.go	Builds CLI args including --model
ProfileResolver	apps/backend/internal/agent/lifecycle/profile_resolver.go	Resolves profile → model
SessionManager	apps/backend/internal/agent/lifecycle/session.go	ACP session init/prompt
TaskSession Model	apps/backend/internal/task/models/models.go	Session persistence
Message Service	apps/backend/internal/task/service/service.go	CreateMessage

---

Requirements Breakdown

UI Requirements

1. Model Selector in Session View: Replace existing agent selector with model dropdown
2. Deferred Application: Selection does NOT take effect immediately
3. Apply on Prompt: Selected model applied when user sends next prompt
4. Persistent State: All subsequent prompts use new model until changed again

Backend Requirements

1. Model Change Detection: Compare current model vs requested model on prompt
2. System Message Insertion: "Model switching to [model name]" message
3. Agent Restart: Stop current agent, restart with new --model flag
4. Session Resumption: Attempt to resume ACP session after restart (if supported)

---

Technical Approach

Phase 1: Frontend Model Selector

1.1 Store State for Pending Model

Add to the Zustand store a per-session "pending model" that tracks what model the user wants to switch to.

Files to modify:

• apps/web/lib/state/store.ts - Add pendingModelBySessionId: Record<string, string>
• apps/web/lib/types/http.ts - Add model-related types if needed

1.2 Model Selector Component

Create a model dropdown in the chat panel that:

• Shows available models for the current agent type
• Shows current model (from session's agent profile)
• Allows selection of a different model
• Stores selection in pendingModelBySessionId[sessionId]

Files to modify:

• apps/web/components/task/task-chat-panel.tsx - Add ModelSelector component
• May create new component: apps/web/components/task/model-selector.tsx

Data source for available models:

• From AvailableAgent.model_config.available_models in the agent discovery store
• Need to resolve which agent the session is using via session.agent_profile_id

1.3 Prompt Submission with Model Override

When user submits a prompt, check if pendingModelBySessionId[sessionId] differs from current model:

• If different: include model field in WebSocket prompt payload
• Clear pending model after submission (backend will handle the switch)

WebSocket payload change:

// Current
{ action: "orchestrator.prompt", payload: { taskId, sessionId, content } }

// Proposed
{ action: "orchestrator.prompt", payload: { taskId, sessionId, content, model?: string } }

Files to modify:

• apps/web/lib/ws/actions.ts - Add model to prompt action payload
• apps/web/components/task/task-chat-panel.tsx - Include model in prompt submission

---

Phase 2: Backend Model Change Detection

2.1 Extend Prompt Handling

Modify the orchestrator's PromptTask to accept an optional model parameter.

Files to modify:

• apps/backend/internal/orchestrator/service.go - PromptTask method
• apps/backend/internal/ws/handlers/orchestrator.go - Parse model from WS payload

Logic flow:

PromptTask(taskId, sessionId, content, model)
    → If model != currentExecution.Model:
        → Insert system message "Switching to {model}..."
        → Stop current agent (graceful shutdown)
        → Update session's effective model
        → Restart agent with new --model flag
        → Resume or reinitialize ACP session
    → Forward prompt to agent

2.2 Executor Model Switch

The executor needs a new method to handle model switching:

Files to modify:

• apps/backend/internal/orchestrator/executor/executor.go
  • Add SwitchModel(sessionId, newModel) method
  • Coordinates stop → rebuild → restart → resume

Implementation sketch:

func (e *Executor) SwitchModel(ctx context.Context, sessionId, newModel string) error {
    exec, ok := e.running[sessionId]
    if !ok {
        return ErrSessionNotFound
    }

    // 1. Stop current agent
    e.lifecycleManager.StopAgent(ctx, exec.AgentID)

    // 2. Update profile with new model
    newProfileInfo := exec.ProfileInfo
    newProfileInfo.Model = newModel

    // 3. Restart agent with new command
    newAgentID, err := e.lifecycleManager.Launch(ctx, LaunchOpts{
        Model: newModel,
        // ... other opts from exec
    })

    // 4. Attempt session resume
    e.lifecycleManager.InitializeSession(ctx, newAgentID, InitOpts{
        IsResume: true,
        ResumeToken: exec.ResumeToken,
    })

    return nil
}

2.3 System Message for Model Switch

Insert a visible message in the chat indicating the model change.

Files to modify:

• apps/backend/internal/task/service/service.go - Use CreateMessage with type status

Message format:

{
  "type": "status",
  "content": "Switching model to Claude Opus 4.5...",
  "metadata": {
    "status_type": "model_switch",
    "from_model": "sonnet4.5",
    "to_model": "opus4.5"
  }
}

---

Phase 3: Session Resumption After Restart

3.1 Agent-Specific Resumption

Different agents handle resumption differently (from agents.json):

Agent	resume_via_acp	resume_flag	can_recover
auggie-agent	false	--resume	false
codex-agent	true	(none)	true

For Codex-style agents (resume_via_acp: true):

• Use ACP session/load with the stored ResumeToken
• Session history is preserved by the agent

For Auggie-style agents (resume_via_acp: false, can_recover: false):

• Cannot truly resume; must reinitialize with fresh ACP session
• Decision: Accept context loss, display system message: "Model switched. A new session was started."

3.2 Handling Resume Failures

Files to modify:

• apps/backend/internal/agent/lifecycle/session.go - InitializeSession error handling

Fallback strategy:

1. Attempt resume with stored token
2. If resume fails → start new session
3. Emit warning message to user: "Session could not be fully resumed after model switch"

---

Integration Points

WebSocket Integration

Layer	Change Required
Client (apps/web/lib/ws/client.ts)	No changes
Actions (apps/web/lib/ws/actions.ts)	Add model field to prompt action
Handler (apps/backend/internal/ws/handlers/orchestrator.go)	Parse model from payload

Wire format example:

{
  "id": "uuid",
  "type": "request",
  "action": "orchestrator.prompt",
  "payload": {
    "task_id": "task-uuid",
    "session_id": "session-uuid",
    "content": "User message",
    "model": "opus4.5"
  }
}

Store Integration

New store slice additions:

// In store.ts
pendingModel: {
  bySessionId: Record<string, string>, // sessionId → pending model id
  setPendingModel: (sessionId: string, modelId: string) => void,
  clearPendingModel: (sessionId: string) => void,
  getPendingModel: (sessionId: string) => string | null,
}

Store → Component data flow:

ModelSelector reads: settingsAgents (for available models), session.agent_profile_id
ModelSelector writes: pendingModel.bySessionId[sessionId]
TaskChatPanel reads: pendingModel.bySessionId[sessionId] on prompt submit

Database Integration

Decision: No schema change (runtime-only)

• Model switch is runtime-only; session's agent_profile_id remains unchanged
• On backend restart, session resumes with original profile model
• No migration required

---

Challenges and Edge Cases

1. Agent Busy During Model Switch Request

Scenario: User selects new model while agent is processing a previous prompt.

Solution options:

• Queue the switch: Store pending model, apply after current response completes
• Block selection: Disable model selector while agent is busy
• Immediate abort: Cancel current response, switch model (destructive)

Recommendation: Queue the switch; apply on next user-initiated prompt.

2. Rapid Model Changes

Scenario: User changes model multiple times before sending a prompt.

Solution: Only the final selection matters; each change overwrites pendingModelBySessionId[sessionId].

3. Resume Failure After Switch

Scenario: Agent restarts with new model but ACP session/load fails.

Handling:

1. Log error for debugging
2. Fall back to new session initialization
3. Show user message: "Model switched successfully but session context could not be fully restored"

4. Different Agent Types

Scenario: User tries to switch to a model not supported by current agent type.

Prevention: Model selector only shows models from agent.model_config.available_models for the session's agent type.

5. Session State Persistence

Scenario: Backend restarts mid-session; what model should resume use?

Behavior: Uses agent_profile_snapshot.model from session creation (original profile model). Runtime model switches are not persisted.

6. Model Availability Changes

Scenario: A model is removed from agent configuration while sessions are active.

Handling: Validate model against current available_models before sending to backend. If invalid, fall back to session's original model.

---

Alternative Approaches

Alternative A: Runtime Model Switching via ACP

Concept: Send a hypothetical session/switch_model ACP call without restarting agent.

Pros:

• No process restart; faster switch
• Seamless context preservation

Cons:

• Not supported by current agent implementations
• Would require agent CLI changes upstream
• Different agents would need different implementations

Verdict: Not viable without significant upstream work.

Alternative B: Model as Separate Session Concept

Concept: Treat each model switch as a "sub-session" within the same UI session.

Pros:

• Clean separation of concerns
• Easy to track model history

Cons:

• Complex UI implications
• Breaks current session model significantly
• Overhead for simple model changes

Verdict: Over-engineering for the requirement.

Alternative C: Pre-select Model Before Session Start Only

Concept: Allow model selection only at session creation, not mid-session.

Pros:

• Simplest implementation
• No restart complexity

Cons:

• Doesn't meet the stated requirement
• Poor UX for iterative work

Verdict: Does not satisfy user requirements.

---

Implementation Phases

Phase 1: Frontend Model Selector (Est. 2-3 days)

Goal: Model dropdown appears in UI; selection stored locally.

Files to modify:

1. apps/web/lib/state/store.ts - Add pendingModel slice
2. apps/web/components/task/task-chat-panel.tsx - Add ModelSelector
3. Create apps/web/components/task/model-selector.tsx - New component

Deliverable: User can select a model, but it has no effect yet.

Phase 2: Backend Prompt Extension (Est. 2-3 days)

Goal: Prompts can include model; backend detects change.

Files to modify:

1. apps/web/lib/ws/actions.ts - Add model to prompt payload
2. apps/backend/internal/ws/handlers/orchestrator.go - Parse model
3. apps/backend/internal/orchestrator/service.go - PromptTask accepts model

Deliverable: Model parameter flows from UI to orchestrator.

Phase 3: Agent Restart Logic (Est. 3-4 days)

Goal: Backend restarts agent with new model when detected.

Files to modify:

1. apps/backend/internal/orchestrator/executor/executor.go - SwitchModel method
2. apps/backend/internal/agent/lifecycle/manager.go - Ensure clean restart
3. apps/backend/internal/task/service/service.go - Insert status message

Deliverable: Model switch triggers agent restart; status message appears.

Phase 4: Session Resumption (Est. 2-3 days)

Goal: Resume ACP session after agent restart (where supported).

Files to modify:

1. apps/backend/internal/agent/lifecycle/session.go - Resume logic
2. apps/backend/internal/orchestrator/executor/executor.go - Handle resume failures

Deliverable: Codex-style agents resume context; Auggie-style gracefully degrade.

Phase 5: Testing & Edge Cases (Est. 2-3 days)

Goal: Verify all scenarios work correctly.

Test scenarios:

• Model switch while agent idle
• Model switch while agent busy (queued)
• Rapid model changes
• Resume failure fallback
• Invalid model selection
• Backend restart mid-session

---

Summary of Files to Modify

File	Changes
apps/web/lib/state/store.ts	Add pendingModel slice
apps/web/components/task/task-chat-panel.tsx	Integrate ModelSelector, pass model on prompt
apps/web/components/task/model-selector.tsx	New component
apps/web/lib/ws/actions.ts	Add model field to prompt action
apps/backend/internal/ws/handlers/orchestrator.go	Parse model from WS payload
apps/backend/internal/orchestrator/service.go	Extend PromptTask signature
apps/backend/internal/orchestrator/executor/executor.go	Add SwitchModel method
apps/backend/internal/agent/lifecycle/manager.go	Ensure clean restart support
apps/backend/internal/agent/lifecycle/session.go	Handle resume after restart
apps/backend/internal/task/service/service.go	Create model switch status message

---

Estimated Total Effort

Phase	Estimate
Phase 1: Frontend Model Selector	2-3 days
Phase 2: Backend Prompt Extension	2-3 days
Phase 3: Agent Restart Logic	3-4 days
Phase 4: Session Resumption	2-3 days
Phase 5: Testing & Edge Cases	2-3 days
Total	11-16 days

---

Resolved Decisions

Decision	Resolution
Database schema	Runtime-only, no schema changes
Non-recoverable agents (Auggie)	Accept context loss, show "new session started" message
Agent busy during switch	Not an issue - model applied on next user prompt
UI placement	Replace existing agent selector with model selector
Confirmation dialog	None required - seamless switch