kandev/docs/plans/session-stream-overload-isolation/plan.md

spec, decision, created, status

spec	decision	created	status
docs/specs/platform/bounded-task-status-delivery.md	docs/decisions/2026-08-02-isolate-replaceable-session-stream-traffic.md	2026-08-02	complete

Implementation Plan: Session Stream Overload Isolation

Overview

Contain a single high-volume ACP session at three independent boundaries while keeping the final transcript exact. Coalesce adjacent assistant/reasoning chunks before the orchestrator performs read/append/write work, replace queued full-message notifications instead of accumulating stale versions, and batch browser store updates once per animation frame. Stabilize Office multi-session membership so task refetches do not replay snapshots. Prove the contract with a deterministic noisy mock-agent scenario on desktop and mobile.

This extends the implemented bounded task-status delivery design. It does not change task summaries, message persistence ownership, or the reserved correlated-response queue.

Incident baseline

• Task 69637afd-c193-4197-aa49-348ac2b0cfb3, session 89f7b247-cfb3-4a4b-8147-8826e13f1630.
• The recorded launcher was opencode-acp using opencode-go/kimi-k3; the logs do not identify this execution as Grok.
• 28,967 unique reasoning-stream events and 31,314 total stream events over about 41 minutes, peaking at 63 reasoning events per second.
• Every chunk crossed lifecycle, orchestrator, message persistence, gateway, JSON, Zustand, and React boundaries.
• Five session.message.updated frames were dropped when the 256-frame notification queue filled. No control-queue overflow or backend crash was recorded.
• Equivalent Office task refetches caused repeated unchanged session unsubscribe/subscribe/snapshot cycles, further amplifying traffic.

Architecture

Lossless lifecycle coalescing

Add an execution-owned coalescer in apps/backend/internal/agent/runtime/lifecycle. It accepts already-normalized assistant/reasoning chunks after legacy newline buffering and protocol ID to Kandev ID mapping. The first non-empty chunk for a record publishes immediately with IsAppend=false. Adjacent later chunks append to one pending segment and flush at the 100 ms cadence with IsAppend=true.

Keep ordering explicit. Before a tool call/result, permission request, completion, error, stream disconnect, prompt-generation reset, execution replacement, or manager shutdown, synchronously detach and publish pending segments before publishing the boundary. Timer callbacks detach under the execution mutex and publish outside it. Teardown stops timers and makes repeat flushes idempotent.

The coalescer operates before handleThinkingStreamingEvent / handleMessageStreamingEvent, so the existing task service naturally performs fewer GetMessage, concatenation, UpdateMessage, and full-message event publications. It does not change stored schemas or final message shapes.

Session-isolated notification scheduling

Replace the client's single best-effort notification channel with a scheduler that retains three classes:

1. the existing control queue for correlated responses/errors;
2. a bounded semantic FIFO for non-replaceable notifications; and
3. bounded per-session queues for replaceable session.message.updated notifications.

The hub passes action and routing identity to the client instead of only raw bytes. For a replaceable message update, derive the key from session_id and message_id. If the key is already pending, replace its bytes without moving its position. A later semantic event for the same session therefore remains behind the newest message state. Drain active session queues round-robin and interleave them with the semantic FIFO after bounded control priority.

Production capacities are named constants, but tests construct small queues. On per-session pressure, evict or reject only queued replaceable entries for that session. Never evict control or semantic notifications to admit a stream replacement. Record content-free counters/log fields for action, client, session, replacement, eviction, and queue depth.

Frontend batching and stable membership

Extract a small replaceable-message scheduler beside lib/ws/handlers/messages.ts. During one animation frame it retains only the newest full payload per (session_id, message_id) and then invokes updateMessage once per changed key. Added/deleted messages and turn-settle events act as barriers: flush or cancel a pending replacement as appropriate so wire order cannot revive deleted state. Clear scheduled work when the WS router is disposed or replaced.

In the Office task detail live-sync hook, derive a stable, sorted session-ID membership key. Refetching equal session IDs or replacing the task object must not rerun subscription cleanup/setup. A genuine membership change applies only the added/removed IDs. The surface remains an intentional multi-session detail view; this task removes churn rather than changing its product scope.

Observability and reconciliation

Lifecycle and gateway logs/metrics must identify pressure without including message content. Existing session snapshot/reconnect and turn-settle behavior remain the reconciliation path; no new user-facing overload banner or protocol action is required.

Tests

Backend

• Use testing/synctest around the lifecycle coalescer. Assert immediate first publication, one append per 100 ms window, exact 30,000-chunk concatenation, independent assistant/reasoning IDs, boundary order, disconnect/shutdown flush, and no timer/goroutine leak.
• Assert the orchestrator/task-service integration sees the reduced event count and stores exact final assistant and thinking content.
• Construct gateway schedulers with tiny capacities. Assert replacement in place, bounded per-session occupancy, round-robin progress, semantic barrier order, control priority, noisy-session-only eviction, metrics, close safety, and zero-value test-client compatibility.

Frontend

• Feed hundreds of session.message.updated payloads for one message in a single fake animation frame. Assert one store update with the newest complete payload, while different message keys each update once.
• Cover add/delete/turn barriers, handler teardown, and separate sessions.
• Rerender Office live sync with new task/session object references but the same IDs and assert no subscription churn; then change membership and assert only the delta is subscribed/unsubscribed.

E2E tests

Add a deterministic mock-agent command that emits a configurable reasoning burst without sleeping or embedding thousands of commands in the prompt. A new overload-isolation Playwright spec opens the noisy session and captures gateway frames while interacting with a second session.

• Desktop Chromium: emit a representative burst, navigate to the quiet task, submit a message, and assert navigation/model controls/responding message remain usable before the noisy run settles.
• Assert server-side burst count was actually produced, the final persisted reasoning text is exact, received replacement frames are materially below source chunks, no control response is lost, and the quiet session progresses.
• Mobile Chrome: repeat the navigation/quiet-session progress assertion through the existing native task-switcher sheet and assert no horizontal overflow.
• Attach source-chunk, lifecycle-publication, gateway replacement/eviction, received-frame, byte, and latency totals. Counts are diagnostics except for the hard boundedness, exact-content, and cross-session-progress assertions.

Mobile design contract

• Desktop outcome / mobile entry: both remain responsive while another session streams. Desktop uses the existing task/sidebar navigation; phone uses the existing task-switcher sheet.
• Nearest exemplar: keep apps/web/components/task/mobile/session-task-switcher-sheet.tsx and shared task/session stores unchanged in layout and navigation ownership.
• Hierarchy / interaction: no new control, gesture, dialog, or route.
• Presentation / scroll: the repair changes transport/state cadence only; existing sheet scrolling, safe areas, and viewport bounds remain intact.
• Shared state: desktop and mobile consume the same batched message and connection state.
• Mobile proof: run the overload scenario under mobile-chrome, switch to a quiet task, verify progress, and assert scrollWidth <= clientWidth.

Implementation waves

Implementation is sequential in the user-started session because Tasks 02 and 03 consume the delivery semantics established by the preceding task. These task files do not authorize subagent work.

• Task 01: Coalesce agent stream ingress
• Task 02: Isolate replaceable session delivery
• Task 03: Bound frontend stream work
• Task 04: Prove overload containment

Verification

All implementation waves are complete. The final verification evidence is:

• go test -race ./internal/agentctl/server/adapter/transport/acp ./internal/agentctl/server/process ./internal/agent/runtime/lifecycle ./internal/gateway/websocket ./cmd/mock-agent — passed (2,633 tests).
• make -C apps/backend test — passed on the final run.
• rtk proxy golangci-lint run ./... — passed (0 issues).
• Focused frontend Vitest — passed (7 tests); web typecheck and lint — passed.
• Desktop Chromium overload E2E — passed (1 test, 13.4s).
• Mobile Chrome overload E2E — passed (1 test, 8.3s).

The E2E evidence confirms exact persisted reasoning, bounded noisy-session frames, quiet-session progress, and desktop/mobile viewport safety. The ACP and process handoff now uses cancellation-aware per-agent backpressure so normalized stream events are not silently discarded before lifecycle coalescing; shutdown cancellation still releases a blocked noisy session.

cd apps/backend && go test ./internal/agent/runtime/lifecycle/... \
  ./internal/orchestrator/... ./internal/task/service/...
cd apps/backend && go test ./internal/gateway/websocket/...
make -C apps/backend test
make -C apps/backend lint
cd apps/web && pnpm exec vitest run \
  lib/ws/handlers/messages.test.ts \
  'app/office/tasks/[id]/use-session-live-sync.test.tsx'
cd apps/web && pnpm run typecheck
cd apps/web && pnpm run lint
cd apps/web && pnpm e2e:run tests/session/session-stream-overload-isolation.spec.ts \
  -- --project=chromium
cd apps/web && pnpm e2e:run tests/session/session-stream-overload-isolation.spec.ts \
  -- --project=mobile-chrome

Risks

• Timer callbacks racing completion could duplicate or reorder content. Detach pending buffers atomically, publish outside locks, and make boundary flushes idempotent.
• Protocol message IDs may interleave. Coalesce only adjacent chunks per typed Kandev record and retain the first-seen queue position.
• A latest-wins gateway queue can reorder a message update past turn completion if replacement positions are not stable. Preserve per-session enqueue order and test semantic barriers.
• Strict control/semantic priority could starve replacement streams. Use bounded priority bursts plus round-robin session scheduling.
• Animation-frame batching can be heavily throttled in background tabs. Keep only complete latest state and rely on selected-session reconciliation when the tab resumes.
• A quiet E2E producer would yield a false pass. Expose and assert the mock burst's produced count before checking isolation.

Out of scope

• Truncating stored assistant or reasoning content.
• Automatically stopping an agent because it streams heavily.
• Splitting sessions across multiple WebSocket connections.
• Changing task-summary fields, task-switcher layout, or message schema.
• Adding a user-facing overload state or runtime feature flag.

Documentation impact

The behavioral contract is added to docs/specs/platform/bounded-task-status-delivery.md and the shared boundary is recorded in docs/decisions/2026-08-02-isolate-replaceable-session-stream-traffic.md. No public docs change is expected because commands, settings, public APIs, and visible interaction patterns do not change.