479 lines
16 KiB
Markdown
479 lines
16 KiB
Markdown
# Dynamic Model Switching Feature Analysis
|
|
|
|
> **Status**: Design Document
|
|
> **Created**: 2026-01-21
|
|
> **Scope**: Mid-session model switching within the Kandev application
|
|
|
|
## Executive Summary
|
|
|
|
This document analyzes the implementation of dynamic model switching, allowing users to change the AI model mid-session without starting a new session. The feature requires UI changes to expose model selection, backend logic to detect model changes and restart agents, and careful handling of session continuity.
|
|
|
|
## Current Architecture Overview
|
|
|
|
### Session Lifecycle (Current Flow)
|
|
|
|
```
|
|
User clicks "Start Session"
|
|
→ Frontend sends orchestrator.start with agentProfileId
|
|
→ Orchestrator.StartTask → Executor.ExecuteWithProfile
|
|
→ Lifecycle Manager resolves profile → gets model from AgentProfileInfo
|
|
→ CommandBuilder.BuildCommand adds --model flag
|
|
→ Agent process started with fixed model
|
|
→ ACP session initialized
|
|
→ User sends prompts via session/prompt
|
|
```
|
|
|
|
**Key Insight**: Model is determined at session start via `AgentProfileInfo.Model` and baked into the agent command. There's no current mechanism to change the model mid-session.
|
|
|
|
### Relevant Components
|
|
|
|
| Component | Path | Role |
|
|
|-----------|------|------|
|
|
| TaskChatPanel | `apps/web/components/task/task-chat-panel.tsx` | Chat UI, displays current model |
|
|
| SessionsDropdown | `apps/web/components/task/sessions-dropdown.tsx` | Session switcher |
|
|
| Orchestrator Service | `apps/backend/internal/orchestrator/service.go` | PromptTask, StartTask, ResumeTaskSession |
|
|
| Executor | `apps/backend/internal/orchestrator/executor/executor.go` | Agent launch, prompt forwarding |
|
|
| Lifecycle Manager | `apps/backend/internal/agent/lifecycle/manager.go` | Agent process management |
|
|
| CommandBuilder | `apps/backend/internal/agent/lifecycle/command.go` | Builds CLI args including --model |
|
|
| ProfileResolver | `apps/backend/internal/agent/lifecycle/profile_resolver.go` | Resolves profile → model |
|
|
| SessionManager | `apps/backend/internal/agent/lifecycle/session.go` | ACP session init/prompt |
|
|
| TaskSession Model | `apps/backend/internal/task/models/models.go` | Session persistence |
|
|
| Message Service | `apps/backend/internal/task/service/service.go` | CreateMessage |
|
|
|
|
---
|
|
|
|
## Requirements Breakdown
|
|
|
|
### UI Requirements
|
|
|
|
1. **Model Selector in Session View**: Replace existing agent selector with model dropdown
|
|
2. **Deferred Application**: Selection does NOT take effect immediately
|
|
3. **Apply on Prompt**: Selected model applied when user sends next prompt
|
|
4. **Persistent State**: All subsequent prompts use new model until changed again
|
|
|
|
### Backend Requirements
|
|
|
|
1. **Model Change Detection**: Compare current model vs requested model on prompt
|
|
2. **System Message Insertion**: "Model switching to [model name]" message
|
|
3. **Agent Restart**: Stop current agent, restart with new `--model` flag
|
|
4. **Session Resumption**: Attempt to resume ACP session after restart (if supported)
|
|
|
|
---
|
|
|
|
## Technical Approach
|
|
|
|
### Phase 1: Frontend Model Selector
|
|
|
|
#### 1.1 Store State for Pending Model
|
|
|
|
Add to the Zustand store a per-session "pending model" that tracks what model the user wants to switch to.
|
|
|
|
**Files to modify:**
|
|
- `apps/web/lib/state/store.ts` - Add `pendingModelBySessionId: Record<string, string>`
|
|
- `apps/web/lib/types/http.ts` - Add model-related types if needed
|
|
|
|
#### 1.2 Model Selector Component
|
|
|
|
Create a model dropdown in the chat panel that:
|
|
- Shows available models for the current agent type
|
|
- Shows current model (from session's agent profile)
|
|
- Allows selection of a different model
|
|
- Stores selection in `pendingModelBySessionId[sessionId]`
|
|
|
|
**Files to modify:**
|
|
- `apps/web/components/task/task-chat-panel.tsx` - Add ModelSelector component
|
|
- May create new component: `apps/web/components/task/model-selector.tsx`
|
|
|
|
**Data source for available models:**
|
|
- From `AvailableAgent.model_config.available_models` in the agent discovery store
|
|
- Need to resolve which agent the session is using via `session.agent_profile_id`
|
|
|
|
#### 1.3 Prompt Submission with Model Override
|
|
|
|
When user submits a prompt, check if `pendingModelBySessionId[sessionId]` differs from current model:
|
|
- If different: include `model` field in WebSocket prompt payload
|
|
- Clear pending model after submission (backend will handle the switch)
|
|
|
|
**WebSocket payload change:**
|
|
|
|
```typescript
|
|
// Current
|
|
{ action: "orchestrator.prompt", payload: { taskId, sessionId, content } }
|
|
|
|
// Proposed
|
|
{ action: "orchestrator.prompt", payload: { taskId, sessionId, content, model?: string } }
|
|
```
|
|
|
|
**Files to modify:**
|
|
- `apps/web/lib/ws/actions.ts` - Add `model` to prompt action payload
|
|
- `apps/web/components/task/task-chat-panel.tsx` - Include model in prompt submission
|
|
|
|
---
|
|
|
|
### Phase 2: Backend Model Change Detection
|
|
|
|
#### 2.1 Extend Prompt Handling
|
|
|
|
Modify the orchestrator's `PromptTask` to accept an optional model parameter.
|
|
|
|
**Files to modify:**
|
|
- `apps/backend/internal/orchestrator/service.go` - `PromptTask` method
|
|
- `apps/backend/internal/ws/handlers/orchestrator.go` - Parse model from WS payload
|
|
|
|
**Logic flow:**
|
|
```
|
|
PromptTask(taskId, sessionId, content, model)
|
|
→ If model != currentExecution.Model:
|
|
→ Insert system message "Switching to {model}..."
|
|
→ Stop current agent (graceful shutdown)
|
|
→ Update session's effective model
|
|
→ Restart agent with new --model flag
|
|
→ Resume or reinitialize ACP session
|
|
→ Forward prompt to agent
|
|
```
|
|
|
|
#### 2.2 Executor Model Switch
|
|
|
|
The executor needs a new method to handle model switching:
|
|
|
|
**Files to modify:**
|
|
- `apps/backend/internal/orchestrator/executor/executor.go`
|
|
- Add `SwitchModel(sessionId, newModel)` method
|
|
- Coordinates stop → rebuild → restart → resume
|
|
|
|
**Implementation sketch:**
|
|
```go
|
|
func (e *Executor) SwitchModel(ctx context.Context, sessionId, newModel string) error {
|
|
exec, ok := e.running[sessionId]
|
|
if !ok {
|
|
return ErrSessionNotFound
|
|
}
|
|
|
|
// 1. Stop current agent
|
|
e.lifecycleManager.StopAgent(ctx, exec.AgentID)
|
|
|
|
// 2. Update profile with new model
|
|
newProfileInfo := exec.ProfileInfo
|
|
newProfileInfo.Model = newModel
|
|
|
|
// 3. Restart agent with new command
|
|
newAgentID, err := e.lifecycleManager.Launch(ctx, LaunchOpts{
|
|
Model: newModel,
|
|
// ... other opts from exec
|
|
})
|
|
|
|
// 4. Attempt session resume
|
|
e.lifecycleManager.InitializeSession(ctx, newAgentID, InitOpts{
|
|
IsResume: true,
|
|
ResumeToken: exec.ResumeToken,
|
|
})
|
|
|
|
return nil
|
|
}
|
|
```
|
|
|
|
#### 2.3 System Message for Model Switch
|
|
|
|
Insert a visible message in the chat indicating the model change.
|
|
|
|
**Files to modify:**
|
|
- `apps/backend/internal/task/service/service.go` - Use `CreateMessage` with type `status`
|
|
|
|
**Message format:**
|
|
```json
|
|
{
|
|
"type": "status",
|
|
"content": "Switching model to Claude Opus 4.5...",
|
|
"metadata": {
|
|
"status_type": "model_switch",
|
|
"from_model": "sonnet4.5",
|
|
"to_model": "opus4.5"
|
|
}
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
### Phase 3: Session Resumption After Restart
|
|
|
|
#### 3.1 Agent-Specific Resumption
|
|
|
|
Different agents handle resumption differently (from `agents.json`):
|
|
|
|
| Agent | `resume_via_acp` | `resume_flag` | `can_recover` |
|
|
|-------|------------------|---------------|---------------|
|
|
| auggie-agent | false | `--resume` | false |
|
|
| codex-agent | true | (none) | true |
|
|
|
|
**For Codex-style agents (resume_via_acp: true):**
|
|
- Use ACP `session/load` with the stored `ResumeToken`
|
|
- Session history is preserved by the agent
|
|
|
|
**For Auggie-style agents (resume_via_acp: false, can_recover: false):**
|
|
- Cannot truly resume; must reinitialize with fresh ACP session
|
|
- **Decision**: Accept context loss, display system message: "Model switched. A new session was started."
|
|
|
|
#### 3.2 Handling Resume Failures
|
|
|
|
**Files to modify:**
|
|
- `apps/backend/internal/agent/lifecycle/session.go` - `InitializeSession` error handling
|
|
|
|
**Fallback strategy:**
|
|
1. Attempt resume with stored token
|
|
2. If resume fails → start new session
|
|
3. Emit warning message to user: "Session could not be fully resumed after model switch"
|
|
|
|
|
|
---
|
|
|
|
## Integration Points
|
|
|
|
### WebSocket Integration
|
|
|
|
| Layer | Change Required |
|
|
|-------|-----------------|
|
|
| Client (`apps/web/lib/ws/client.ts`) | No changes |
|
|
| Actions (`apps/web/lib/ws/actions.ts`) | Add `model` field to prompt action |
|
|
| Handler (`apps/backend/internal/ws/handlers/orchestrator.go`) | Parse `model` from payload |
|
|
|
|
**Wire format example:**
|
|
```json
|
|
{
|
|
"id": "uuid",
|
|
"type": "request",
|
|
"action": "orchestrator.prompt",
|
|
"payload": {
|
|
"task_id": "task-uuid",
|
|
"session_id": "session-uuid",
|
|
"content": "User message",
|
|
"model": "opus4.5"
|
|
}
|
|
}
|
|
```
|
|
|
|
### Store Integration
|
|
|
|
**New store slice additions:**
|
|
|
|
```typescript
|
|
// In store.ts
|
|
pendingModel: {
|
|
bySessionId: Record<string, string>, // sessionId → pending model id
|
|
setPendingModel: (sessionId: string, modelId: string) => void,
|
|
clearPendingModel: (sessionId: string) => void,
|
|
getPendingModel: (sessionId: string) => string | null,
|
|
}
|
|
```
|
|
|
|
**Store → Component data flow:**
|
|
```
|
|
ModelSelector reads: settingsAgents (for available models), session.agent_profile_id
|
|
ModelSelector writes: pendingModel.bySessionId[sessionId]
|
|
TaskChatPanel reads: pendingModel.bySessionId[sessionId] on prompt submit
|
|
```
|
|
|
|
### Database Integration
|
|
|
|
**Decision: No schema change (runtime-only)**
|
|
- Model switch is runtime-only; session's `agent_profile_id` remains unchanged
|
|
- On backend restart, session resumes with original profile model
|
|
- No migration required
|
|
|
|
---
|
|
|
|
## Challenges and Edge Cases
|
|
|
|
### 1. Agent Busy During Model Switch Request
|
|
|
|
**Scenario**: User selects new model while agent is processing a previous prompt.
|
|
|
|
**Solution options:**
|
|
- **Queue the switch**: Store pending model, apply after current response completes
|
|
- **Block selection**: Disable model selector while agent is busy
|
|
- **Immediate abort**: Cancel current response, switch model (destructive)
|
|
|
|
**Recommendation**: Queue the switch; apply on next user-initiated prompt.
|
|
|
|
### 2. Rapid Model Changes
|
|
|
|
**Scenario**: User changes model multiple times before sending a prompt.
|
|
|
|
**Solution**: Only the final selection matters; each change overwrites `pendingModelBySessionId[sessionId]`.
|
|
|
|
### 3. Resume Failure After Switch
|
|
|
|
**Scenario**: Agent restarts with new model but ACP session/load fails.
|
|
|
|
**Handling:**
|
|
1. Log error for debugging
|
|
2. Fall back to new session initialization
|
|
3. Show user message: "Model switched successfully but session context could not be fully restored"
|
|
|
|
### 4. Different Agent Types
|
|
|
|
**Scenario**: User tries to switch to a model not supported by current agent type.
|
|
|
|
**Prevention**: Model selector only shows models from `agent.model_config.available_models` for the session's agent type.
|
|
|
|
### 5. Session State Persistence
|
|
|
|
**Scenario**: Backend restarts mid-session; what model should resume use?
|
|
|
|
**Behavior**: Uses `agent_profile_snapshot.model` from session creation (original profile model). Runtime model switches are not persisted.
|
|
|
|
### 6. Model Availability Changes
|
|
|
|
**Scenario**: A model is removed from agent configuration while sessions are active.
|
|
|
|
**Handling**: Validate model against current `available_models` before sending to backend. If invalid, fall back to session's original model.
|
|
|
|
---
|
|
|
|
## Alternative Approaches
|
|
|
|
### Alternative A: Runtime Model Switching via ACP
|
|
|
|
**Concept**: Send a hypothetical `session/switch_model` ACP call without restarting agent.
|
|
|
|
**Pros:**
|
|
- No process restart; faster switch
|
|
- Seamless context preservation
|
|
|
|
**Cons:**
|
|
- Not supported by current agent implementations
|
|
- Would require agent CLI changes upstream
|
|
- Different agents would need different implementations
|
|
|
|
**Verdict**: Not viable without significant upstream work.
|
|
|
|
### Alternative B: Model as Separate Session Concept
|
|
|
|
**Concept**: Treat each model switch as a "sub-session" within the same UI session.
|
|
|
|
**Pros:**
|
|
- Clean separation of concerns
|
|
- Easy to track model history
|
|
|
|
**Cons:**
|
|
- Complex UI implications
|
|
- Breaks current session model significantly
|
|
- Overhead for simple model changes
|
|
|
|
**Verdict**: Over-engineering for the requirement.
|
|
|
|
### Alternative C: Pre-select Model Before Session Start Only
|
|
|
|
**Concept**: Allow model selection only at session creation, not mid-session.
|
|
|
|
**Pros:**
|
|
- Simplest implementation
|
|
- No restart complexity
|
|
|
|
**Cons:**
|
|
- Doesn't meet the stated requirement
|
|
- Poor UX for iterative work
|
|
|
|
**Verdict**: Does not satisfy user requirements.
|
|
|
|
---
|
|
|
|
## Implementation Phases
|
|
|
|
### Phase 1: Frontend Model Selector (Est. 2-3 days)
|
|
|
|
**Goal**: Model dropdown appears in UI; selection stored locally.
|
|
|
|
**Files to modify:**
|
|
1. `apps/web/lib/state/store.ts` - Add pendingModel slice
|
|
2. `apps/web/components/task/task-chat-panel.tsx` - Add ModelSelector
|
|
3. Create `apps/web/components/task/model-selector.tsx` - New component
|
|
|
|
**Deliverable**: User can select a model, but it has no effect yet.
|
|
|
|
### Phase 2: Backend Prompt Extension (Est. 2-3 days)
|
|
|
|
**Goal**: Prompts can include model; backend detects change.
|
|
|
|
**Files to modify:**
|
|
1. `apps/web/lib/ws/actions.ts` - Add model to prompt payload
|
|
2. `apps/backend/internal/ws/handlers/orchestrator.go` - Parse model
|
|
3. `apps/backend/internal/orchestrator/service.go` - PromptTask accepts model
|
|
|
|
**Deliverable**: Model parameter flows from UI to orchestrator.
|
|
|
|
### Phase 3: Agent Restart Logic (Est. 3-4 days)
|
|
|
|
**Goal**: Backend restarts agent with new model when detected.
|
|
|
|
**Files to modify:**
|
|
1. `apps/backend/internal/orchestrator/executor/executor.go` - SwitchModel method
|
|
2. `apps/backend/internal/agent/lifecycle/manager.go` - Ensure clean restart
|
|
3. `apps/backend/internal/task/service/service.go` - Insert status message
|
|
|
|
**Deliverable**: Model switch triggers agent restart; status message appears.
|
|
|
|
### Phase 4: Session Resumption (Est. 2-3 days)
|
|
|
|
**Goal**: Resume ACP session after agent restart (where supported).
|
|
|
|
**Files to modify:**
|
|
1. `apps/backend/internal/agent/lifecycle/session.go` - Resume logic
|
|
2. `apps/backend/internal/orchestrator/executor/executor.go` - Handle resume failures
|
|
|
|
**Deliverable**: Codex-style agents resume context; Auggie-style gracefully degrade.
|
|
|
|
### Phase 5: Testing & Edge Cases (Est. 2-3 days)
|
|
|
|
**Goal**: Verify all scenarios work correctly.
|
|
|
|
**Test scenarios:**
|
|
- Model switch while agent idle
|
|
- Model switch while agent busy (queued)
|
|
- Rapid model changes
|
|
- Resume failure fallback
|
|
- Invalid model selection
|
|
- Backend restart mid-session
|
|
|
|
---
|
|
|
|
## Summary of Files to Modify
|
|
|
|
| File | Changes |
|
|
|------|---------|
|
|
| `apps/web/lib/state/store.ts` | Add pendingModel slice |
|
|
| `apps/web/components/task/task-chat-panel.tsx` | Integrate ModelSelector, pass model on prompt |
|
|
| `apps/web/components/task/model-selector.tsx` | New component |
|
|
| `apps/web/lib/ws/actions.ts` | Add model field to prompt action |
|
|
| `apps/backend/internal/ws/handlers/orchestrator.go` | Parse model from WS payload |
|
|
| `apps/backend/internal/orchestrator/service.go` | Extend PromptTask signature |
|
|
| `apps/backend/internal/orchestrator/executor/executor.go` | Add SwitchModel method |
|
|
| `apps/backend/internal/agent/lifecycle/manager.go` | Ensure clean restart support |
|
|
| `apps/backend/internal/agent/lifecycle/session.go` | Handle resume after restart |
|
|
| `apps/backend/internal/task/service/service.go` | Create model switch status message |
|
|
|
|
---
|
|
|
|
## Estimated Total Effort
|
|
|
|
| Phase | Estimate |
|
|
|-------|----------|
|
|
| Phase 1: Frontend Model Selector | 2-3 days |
|
|
| Phase 2: Backend Prompt Extension | 2-3 days |
|
|
| Phase 3: Agent Restart Logic | 3-4 days |
|
|
| Phase 4: Session Resumption | 2-3 days |
|
|
| Phase 5: Testing & Edge Cases | 2-3 days |
|
|
| **Total** | **11-16 days** |
|
|
|
|
---
|
|
|
|
## Resolved Decisions
|
|
|
|
| Decision | Resolution |
|
|
|----------|------------|
|
|
| Database schema | Runtime-only, no schema changes |
|
|
| Non-recoverable agents (Auggie) | Accept context loss, show "new session started" message |
|
|
| Agent busy during switch | Not an issue - model applied on next user prompt |
|
|
| UI placement | Replace existing agent selector with model selector |
|
|
| Confirmation dialog | None required - seamless switch |
|
|
|