João Salavisa 1fdeda904b feat: add voice mode for chat input (#1159)
* feat: add voice mode for chat input

Adds a voice input button to the chat composer with three transcription engines:
in-browser Web Speech API, in-browser Whisper via @huggingface/transformers, and
a server-side OpenAI Whisper fallback gated on KANDEV_VOICE_OPENAI_API_KEY.

User-configurable from a new /settings/voice-mode page:
- Master on/off toggle (defaults to on; opt-out, not opt-in)
- Engine preference (auto / Web Speech / Whisper Web / Whisper Server)
- Language hint (auto-detect or BCP-47 pinned)
- Activation mode (click-to-toggle or hold-to-talk)
- Auto-send after transcription
- Whisper Web model size (tiny / base / small)
- Editable Cmd+Shift+M keyboard shortcut

Voice settings live under user_settings.voice_mode as a nested JSON object —
no schema migration. Setting changes broadcast via the existing
user.settings.updated WS event so any open chat tab updates live.

The mic button is rendered on the right of the composer alongside the send
button (round primary fill; destructive fill with pulse ring while recording).
On browsers without a usable engine (e.g. iOS Chrome over HTTP) the button
stays visible as a muted greyed icon — tapping it surfaces a toast with the
reason (typically a missing secure context).

Whisper Web runs in a Web Worker with fp16 decoder weights (avoids the
MatMulNBits scale-missing crash that q4/q8 hit on WASM). Models are pulled
from the onnx-community/* mirror and cached by transformers.js in the Cache
Storage on first use.

* fix: address voice mode PR review feedback and E2E selector clash

Bot-review fixes from #1159:
- Map http.MaxBytesError from c.FormFile to 413 instead of bare 400.
- Strip BCP-47 region suffix before passing language hint to Whisper Web.
- Recover from worker crash: terminate + null worker + reset ready flag so
  next init/transcribe creates a fresh worker instead of hanging.
- Read live store state via storeApi.getState() in voice-mode settings
  savers to avoid stale-closure clobbering concurrent settings updates.
- Detach Web Speech callbacks before abort() so trailing onend/onerror
  events don't mutate hook state after cancel().
- Spread caller init before method/body in voice-api fetch so a caller-
  supplied init can't override the multipart POST.
- Replace t.Fatalf with t.Errorf+return inside test HTTP handler goroutine
  (FailNow from non-test goroutine is unsafe).
- Add Enabled/AutoSend zeroing assertions in partial-update test so the
  bool-overwrite-on-PATCH behavior is locked in explicitly.

E2E fix:
- toolbar-overflow spec used button.rounded-full which now matches both
  the send and voice buttons (strict-mode violation). Switched to the
  submit-message-button testid for an unambiguous locator.

* fix: scope voice settings rollback + whisper worker error to changed keys

Round 2 of PR review feedback on #1159:
- voice-mode settings rollback now restores only the keys patched by the
  failing request (not the whole snapshot), preserving concurrent edits
  to unrelated voiceMode fields.
- Shortcut rollback diffs previous vs next and reverts only the changed
  keys (restore prior value or delete if absent), preserving other
  concurrent shortcut edits.
- Whisper worker error handler captures the worker reference at attach
  time and only clears refs when this.worker still matches it — a stale
  error from a previously replaced worker no longer nukes the active one.
- Update stale whisper-web-models JSDoc that still pointed at Xenova
  mirrors.

* fix: voice mode progress scale, hold-mode race, race-aware rollback

Round 3 of PR review feedback on #1159:
- Normalise Whisper progress to 0–1 in ensureWhisperClient (transformers.js
  emits 0–100, the button's display + the `ready: 1` convention expect
  0–1 — fixes the 5000% mid-download display).
- Claim driverRef synchronously at the top of finishCapture so concurrent
  pointerup + pointerleave invocations early-return instead of racing and
  clobbering a freshly-started recording's ref.
- Race-aware rollback: voice-mode and shortcut catch branches now skip
  reverting a key when latest[key] !== next[key]. A failed earlier request
  no longer overwrites a later successful save to the same key.
- Update the Worker wire-protocol comment to show onnx-community/whisper-base
  instead of the stale Xenova/* example.

---------

Co-authored-by: Kandev Agent <agent@kandev.dev>
2026-06-01 14:32:39 +01:00
2026-01-16 01:59:19 +00:00
2026-05-25 10:04:40 +01:00
2026-05-31 21:35:11 +00:00
2026-02-17 09:19:42 +00:00
2026-05-01 21:02:41 +01:00

Kandev

Manage and run tasks in parallel. Orchestrate agents. Review changes. Ship value.

Workflows | Run as a Service | Roadmap | Contributing | Architecture | Discord

Kandev Demo

See all screenshots

What

Organize work across kanban and pipeline views with opinionated workflows and execute multiple tasks in parallel. Assign agents from any provider, and review their output in an integrated workspace - file editor, file tree, terminal, browser preview, and git changes in one place. Terminal agent TUIs are great for running agents, but reviewing and iterating on changes there doesn't scale.

Run it locally or self-host it on your own infrastructure and access it from anywhere via Tailscale or any VPN.

Open source, multi-provider, no telemetry, not tied to any cloud.

Vision

Humans stay in control. Define tasks, build agentic workflows with gates, review every change, decide what ships.

  • Review-first - Humans support production systems. We need to understand (yet) and trust the code that gets deployed.
  • Your workflow - Every team is different, and not every developer uses AI the same way. Define workflows once, share them across the team, and give everyone a consistent process for working with agents - regardless of experience level.
  • Remote agents - Running multiple agents on a large codebase can quickly saturate a local machine. The goal is a single control plane: offload execution to servers, orchestrate from anywhere, including your phone.

Features

  • Multi-agent support - Claude Code, Codex, GitHub Copilot, Gemini CLI, Amp, Auggie, OpenCode, Cursor, Qwen, Factory Droid, iFlow, Kilocode, Pi, Kimi, AWS Kiro, Qoder, Trae
  • Parallel task execution – start and manage multiple tasks from different sources simultaneously, boosting productivity with AI agents
  • Integrated workspace - Built-in terminal, code editor with LSP, git changes panel, embedded vscode and chat in one IDE-like view
  • Kanban task management - Drag-and-drop boards, columns, and workflow automation
  • Agentic workflows - Multi-step pipelines that mix-and-match agents per step - for example, Claude Code Opus to design a plan, GitHub Copilot Sonnet to implement it, and Codex GPT 5.4 to review the changes. See docs/workflows.md
  • Sub-tasks - Agents can spawn sub-tasks that resume from the parent task's session. Useful for splitting a task that has grown too big, or producing several PRs from the same starting point.
  • CLI passthrough - Drop into raw agent CLI mode for direct terminal interaction with any supported agent, leverage the full power of their TUIs
  • Workspace isolation - Git worktrees prevent concurrent agents from conflicting
  • Multi-repository tasks - Span a single task across multiple repositories, with one worktree per repo, per-repo branches, per-repo PRs, and per-repo grouping in the Changes panel and review dialog
  • Flexible runtimes - Run agents as local processes, in isolated Docker containers or in remote executors like sprites.dev
  • Session management - Resume and review agent conversations
  • Stats - Track your productivity with stats on the completed tasks, agent turns, etc

Integrations

GitHub Jira Linear

Connect Kandev to the tools your team already uses — pull issues into the kanban, link tasks to PRs, and surface review activity inline.

Supported ACP Agents

Agent Launch
Claude Code npx -y @agentclientprotocol/claude-agent-acp
Codex npx -y @zed-industries/codex-acp
GitHub Copilot npx -y @github/copilot --acp
Gemini CLI npx -y @google/gemini-cli --acp
Amp npx -y amp-acp
Auggie npx -y @augmentcode/auggie --acp
OpenCode opencode acp
Cursor cursor-agent acp (requires Cursor Pro)
Qwen npx -y @qwen-code/qwen-code --acp
Factory Droid npx -y droid exec --output-format acp
iFlow (beta) npx -y @iflow-ai/iflow-cli --experimental-acp
Kilocode npx -y @kilocode/cli acp
Pi npx -y pi-acp
Kimi kimi acp (install Kimi CLI from Moonshot AI)
Kiro kiro-cli-chat acp (install Kiro CLI from AWS)
Qoder qodercli --acp (install Qoder CLI)
Trae traecli acp serve (install Trae IDE CLI)

All agents communicate via ACP (Agent Client Protocol). Some agents support ACP natively, while others use ACP adapter packages that bridge their native protocols. CLI Passthrough mode is also available for direct terminal interaction with any agent CLI. If your agent isn't supported yet, open an issue or submit a PR with the integration. See Adding a New Agent CLI for a step-by-step guide.

Bring your own TUI agents

There is support for running any agent as TUI inside a terminal. Just add the cli command in the agent profile settings and the task will start the agent inside a PTY terminal instead of using ACP.

Supported Executors

Executor Description
Local Process Runs the agent as a local process on the host machine
Docker Runs the agent in an isolated Docker container
Sprites Runs the agent in a remote cloud environment via sprites.dev

Each executor uses git worktrees for workspace isolation, preventing concurrent agents from conflicting.

Quick Start

Homebrew (macOS, Linux)

brew install kdlbs/kandev/kandev
kandev

NPX

npx kandev@latest

Requires npm 7+. Installs the CLI plus a platform-matched runtime bundle in one step.

NPM (global)

npm install -g kandev@latest
kandev

Updates

  • brew upgrade kandev
  • npx kandev@latest (always uses the latest published version)
  • npm install -g kandev@latest

The worktrees and sqlite db live in ~/.kandev by default. Works on macOS, Linux, and Windows (WSL or native).

From Source

# Clone the repository
git clone git@github.com:kdlbs/kandev.git
cd kandev

# Start in production mode
make start

Prerequisites: Go 1.26+, Node.js 18+, pnpm, Docker (optional - needed for container runtimes)

High level architecture

graph LR
    UI[Web UI] --> Backend


        Backend["Backend (Orchestrator)"]


    Backend --> W1 & W2

    subgraph W1[Local Process]
        Agent1[Agent CLI] --- WT1[Worktree]
    end

    subgraph W2[Docker Container]
        Agent2[Agent CLI] --- WT2[Worktree]
    end

We also want to add support for these remote runtimes:

  • Remote SSH - run agents on remote servers over SSH, using docker or local processes with workspace isolation
  • K8s operator - run agents in a Kubernetes cluster, with auto-scaling and resource management.
Development

Project Structure

apps/
├── backend/    # Go backend (orchestrator, lifecycle, agentctl, WS gateway)
├── web/        # Next.js frontend (SSR, Zustand, real-time subscriptions)
├── cli/        # CLI tool (npx kandev launcher)
└── packages/   # Shared UI components & types

Prerequisites

  • Go 1.21+
  • Node.js 18+
  • pnpm
  • Docker (optional)

Running Dev Servers

# Start everything (backend + frontend with auto ports)
make dev

# Or run separately
make dev-backend    # Backend on :38429
make dev-web        # Frontend on :37429

Testing & Linting

make test           # Run all tests (backend + web)
make lint           # Run all linters
make typecheck      # TypeScript type checking
make fmt            # Format all code

Pre-commit Hooks

# Install pre-commit (https://pre-commit.com/#install)
pipx install pre-commit

# Install git hooks
pre-commit install

Comparison to Other Tools

There are a few similar tools in this space, and new ones appearing everyday. Here's what sets this one apart:

  • Server-first architecture - Not a desktop app. Runs as a server you can access from any device, including your phone. Start a task away from your computer and check in on it later.
  • Remote runtimes - Run agents on remote servers and Docker hosts, not just your local machine.
  • Multi-provider - Use Claude Code, Codex, Copilot, Gemini, Amp, Auggie, OpenCode, Cursor, Qwen, Droid, iFlow, Kilocode, Pi, Kimi, Kiro, Qoder, and Trae side by side. Not locked to one vendor.
  • CLI passthrough and chat - Interact with agents through structured chat messages or drop into raw CLI mode for full agent TUI capabilities.
  • Open source and self-hostable - No vendor lock-in, no telemetry, runs on your infrastructure.

Contributing

Contributions are welcome! Please read CONTRIBUTING.md before opening a PR.

See the issue tracker for open tasks, or join our Discord to chat with maintainers and other contributors.

Acknowledgments

Built with these excellent open-source projects:

Monaco Editor · Tiptap · xterm.js · dockview · CodeMirror · dnd-kit · Mermaid · Recharts · TanStack Table · Zustand · Shadcn/UI · Radix UI · Tailwind CSS

License

AGPL-3.0

Star History

Star History Chart
S
Description
Upstream mirror of https://github.com/kdlbs/kandev (main). Internal development and testing.
Readme 371 MiB
Languages
Go 58.3%
TypeScript 37.6%
HTML 1.9%
JavaScript 0.8%
Python 0.7%
Other 0.5%