fix(web): fix hold-to-talk on mobile and add coarse-pointer toggle fallback (#1231)
* fix(web): make voice-mode hold gesture survive touch and add mobile coverage
Mobile users with the chat composer's mic button in hold-to-talk mode lost
their recording the moment a finger crept off the 28px target, because
`buildHoldHandlers` aborted on `pointerleave`/`pointercancel` without ever
claiming the pointer.
- `buildHoldHandlers` now claims the pointer on `pointerdown` via
`setPointerCapture` and releases it on `pointerup`/`pointercancel`. The
`pointerleave` handler is dropped — capture keeps the gesture alive across
small finger movement, OS gesture handoff, and soft-keyboard reflows.
- `EnabledVoiceInputButton` resolves an `effectiveMode` via a coarse-pointer
check (`matchMedia("(pointer: coarse)")`); when the stored preference is
`hold` and the device is touch-first, the mic falls back to toggle so a
bad stored preference can't trap users in a broken UX. Persisted
`voiceMode.mode` is left untouched so docking a keyboard restores the
user's choice without a save.
- The button bumps to 40×40 on coarse-pointer devices to meet the 44pt touch-
target floor without disturbing the desktop submit/voice pair.
- The unsupported fallback path (`UnsupportedVoiceButton`) is unchanged so
LAN-HTTP mobile users still get the HTTPS explanation toast.
Adds `voice-input-button.test.tsx` covering capture/release on pointer
events, the absence of `pointerleave` regression, the coarse-pointer toggle
override, and the touch-target class swap.
Adds `mobile-voice-mode.spec.ts` (Playwright mobile-chrome / Pixel 5)
verifying the Voice Mode settings page is reachable from the mobile sidebar
trigger and that the mic button mounts with `data-effective-mode="toggle"`
on the mobile chat composer.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* test(web): drop unused testPage param from mobile-voice-mode seedTask helper
The helper only calls apiClient and does not touch the Playwright Page
object — the parameter was carried over from the neighbour spec's pattern
where the helper also navigates. Removing it also drops the now-unused
`Page` type import.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* refactor(web): drop redundant Element type cast from safePointerRelease
`Element.hasPointerCapture` and `releasePointerCapture` are already declared
in lib.dom.d.ts, so the `target as Element & { hasPointerCapture?: ... }`
casts were over-defensive. Replace with a direct `typeof` runtime check,
which still guards against the headless DOMs that strip these methods.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* test(web): assert coarse-pointer touch-target class in mobile voice spec
The previous assertion (`data-effective-mode === "toggle"`) would have
passed even if the coarse-pointer override were reverted, since the default
stored `voiceMode.mode` is already `"toggle"`. Add a check on the `h-10
w-10` class swap — that class pair is only emitted when
`useIsCoarsePointer()` returned true, so the test now genuinely exercises
the override on the e2e surface.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* test(web): extend voice-input-button unit coverage for toggle paths
- Toggle-mode + fine-pointer: pointerdown must not start recording; click
drives the toggle. Guards against an accidental future change that wires
hold pointer handlers when stored mode is "toggle".
- Recording state + click: stop() fires (not start()), exercising the
buildToggleHandler branch the original suite skipped.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* test(web): seed hold mode in E2E and fix misleading test title
- Add `voice_mode` field to `saveUserSettings` in api-client.ts so E2E
tests can PATCH the user's stored activation mode via the API.
- Seed `mode: "hold"` before the mic-button assertion so
`data-mode="hold"` + `data-effective-mode="toggle"` together prove the
coarse-pointer override fired (not that the default was already toggle).
- Rename settings-page test to drop the "from mobile sidebar" claim —
the test navigates directly to the URL; sidebar existence is still
asserted as a regression guard.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Kandev Agent <agent@kandev.dev>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,245 @@
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import { cleanup, fireEvent, render, screen } from "@testing-library/react";
|
||||
|
||||
// ── Hoisted mocks (must be set before importing the component under test) ──
|
||||
|
||||
const voicePrefs = vi.hoisted(() => ({
|
||||
value: {
|
||||
enabled: true,
|
||||
engine: "auto" as "auto" | "webSpeech" | "whisperWeb" | "whisperServer",
|
||||
language: "auto",
|
||||
mode: "hold" as "hold" | "toggle",
|
||||
autoSend: false,
|
||||
whisperWebModel: "base" as "tiny" | "base" | "small",
|
||||
},
|
||||
}));
|
||||
|
||||
const voiceInputResult = vi.hoisted(() => ({
|
||||
supported: true as boolean,
|
||||
state: "idle" as "idle" | "requesting" | "recording" | "processing",
|
||||
modelLoad: { state: "idle" as const, progress: 0 },
|
||||
start: vi.fn(async () => undefined),
|
||||
stop: vi.fn(async () => undefined),
|
||||
cancel: vi.fn(() => undefined),
|
||||
engine: "webSpeech" as const,
|
||||
error: null as null | { code: string; message: string },
|
||||
}));
|
||||
|
||||
const coarsePointer = vi.hoisted(() => ({ value: false }));
|
||||
|
||||
vi.mock("@/components/state-provider", () => ({
|
||||
useAppStore: (
|
||||
selector: (state: {
|
||||
userSettings: {
|
||||
voiceMode: typeof voicePrefs.value;
|
||||
keyboardShortcuts: Record<string, unknown>;
|
||||
};
|
||||
}) => unknown,
|
||||
) =>
|
||||
selector({
|
||||
userSettings: { voiceMode: voicePrefs.value, keyboardShortcuts: {} },
|
||||
}),
|
||||
}));
|
||||
|
||||
vi.mock("@/hooks/use-voice-input", () => ({
|
||||
useVoiceInput: () => voiceInputResult,
|
||||
}));
|
||||
|
||||
vi.mock("@/hooks/use-keyboard-shortcut", () => ({
|
||||
useKeyboardShortcut: () => undefined,
|
||||
}));
|
||||
|
||||
vi.mock("@/components/toast-provider", () => ({
|
||||
useToast: () => ({ toast: vi.fn() }),
|
||||
}));
|
||||
|
||||
vi.mock("@kandev/ui/tooltip", () => ({
|
||||
Tooltip: ({ children }: { children: React.ReactNode }) => <>{children}</>,
|
||||
TooltipTrigger: ({ children }: { children: React.ReactNode }) => <>{children}</>,
|
||||
TooltipContent: ({ children }: { children: React.ReactNode }) => <>{children}</>,
|
||||
}));
|
||||
|
||||
// ── matchMedia stub honouring the hoisted `coarsePointer.value` flag ──
|
||||
|
||||
beforeEach(() => {
|
||||
voicePrefs.value = {
|
||||
enabled: true,
|
||||
engine: "auto",
|
||||
language: "auto",
|
||||
mode: "hold",
|
||||
autoSend: false,
|
||||
whisperWebModel: "base",
|
||||
};
|
||||
voiceInputResult.supported = true;
|
||||
voiceInputResult.state = "idle";
|
||||
voiceInputResult.modelLoad = { state: "idle", progress: 0 };
|
||||
voiceInputResult.start.mockClear();
|
||||
voiceInputResult.stop.mockClear();
|
||||
voiceInputResult.cancel.mockClear();
|
||||
coarsePointer.value = false;
|
||||
|
||||
Object.defineProperty(window, "matchMedia", {
|
||||
configurable: true,
|
||||
writable: true,
|
||||
value: (query: string) => ({
|
||||
matches: query.includes("coarse") ? coarsePointer.value : false,
|
||||
media: query,
|
||||
onchange: null,
|
||||
addEventListener: () => undefined,
|
||||
removeEventListener: () => undefined,
|
||||
addListener: () => undefined,
|
||||
removeListener: () => undefined,
|
||||
dispatchEvent: () => false,
|
||||
}),
|
||||
});
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
cleanup();
|
||||
});
|
||||
|
||||
// Re-import after the mocks so the module under test sees the mocked deps.
|
||||
import { VoiceInputButton } from "./voice-input-button";
|
||||
|
||||
const BUTTON_TESTID = "voice-input-button";
|
||||
|
||||
function renderButton() {
|
||||
return render(<VoiceInputButton onTranscript={() => {}} onAutoSend={() => {}} />);
|
||||
}
|
||||
|
||||
describe("VoiceInputButton — hold-mode pointer handlers", () => {
|
||||
it("captures the pointer on pointerdown and starts recording", () => {
|
||||
renderButton();
|
||||
const button = screen.getByTestId(BUTTON_TESTID);
|
||||
const setCap = vi.fn();
|
||||
(button as unknown as { setPointerCapture: (id: number) => void }).setPointerCapture = setCap;
|
||||
|
||||
fireEvent.pointerDown(button, { pointerId: 7 });
|
||||
|
||||
expect(setCap).toHaveBeenCalledWith(7);
|
||||
expect(voiceInputResult.start).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it("releases the pointer and stops on pointerup", () => {
|
||||
renderButton();
|
||||
const button = screen.getByTestId(BUTTON_TESTID);
|
||||
const releaseCap = vi.fn();
|
||||
(button as unknown as { hasPointerCapture: () => boolean }).hasPointerCapture = () => true;
|
||||
(button as unknown as { releasePointerCapture: (id: number) => void }).releasePointerCapture =
|
||||
releaseCap;
|
||||
|
||||
fireEvent.pointerUp(button, { pointerId: 7 });
|
||||
|
||||
expect(releaseCap).toHaveBeenCalledWith(7);
|
||||
expect(voiceInputResult.stop).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it("stops on pointercancel without triggering on pointerleave", () => {
|
||||
renderButton();
|
||||
const button = screen.getByTestId(BUTTON_TESTID);
|
||||
(button as unknown as { hasPointerCapture: () => boolean }).hasPointerCapture = () => false;
|
||||
(button as unknown as { releasePointerCapture: (id: number) => void }).releasePointerCapture =
|
||||
() => undefined;
|
||||
|
||||
// pointerleave used to abort recording. Confirm it no longer does.
|
||||
fireEvent.pointerLeave(button, { pointerId: 7 });
|
||||
expect(voiceInputResult.stop).not.toHaveBeenCalled();
|
||||
|
||||
fireEvent.pointerCancel(button, { pointerId: 7 });
|
||||
expect(voiceInputResult.stop).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it("does not throw if setPointerCapture is not present on the element", () => {
|
||||
renderButton();
|
||||
const button = screen.getByTestId(BUTTON_TESTID);
|
||||
// Some test/UA combinations strip setPointerCapture — handler must swallow.
|
||||
expect(() => fireEvent.pointerDown(button, { pointerId: 1 })).not.toThrow();
|
||||
});
|
||||
});
|
||||
|
||||
describe("VoiceInputButton — coarse-pointer override", () => {
|
||||
it("reports stored hold mode but effective toggle when matchMedia(coarse) matches", () => {
|
||||
coarsePointer.value = true;
|
||||
renderButton();
|
||||
const button = screen.getByTestId(BUTTON_TESTID);
|
||||
expect(button.getAttribute("data-mode")).toBe("hold");
|
||||
expect(button.getAttribute("data-effective-mode")).toBe("toggle");
|
||||
});
|
||||
|
||||
it("does not attach hold pointer handlers in coarse-pointer toggle fallback", () => {
|
||||
coarsePointer.value = true;
|
||||
renderButton();
|
||||
const button = screen.getByTestId(BUTTON_TESTID);
|
||||
fireEvent.pointerDown(button, { pointerId: 1 });
|
||||
fireEvent.pointerUp(button, { pointerId: 1 });
|
||||
// toggle handler is bound to onClick; pointer events alone should be inert.
|
||||
expect(voiceInputResult.start).not.toHaveBeenCalled();
|
||||
expect(voiceInputResult.stop).not.toHaveBeenCalled();
|
||||
|
||||
// Click is what drives toggle.
|
||||
fireEvent.click(button);
|
||||
expect(voiceInputResult.start).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it("preserves hold behaviour on fine-pointer devices", () => {
|
||||
coarsePointer.value = false;
|
||||
renderButton();
|
||||
const button = screen.getByTestId(BUTTON_TESTID);
|
||||
expect(button.getAttribute("data-effective-mode")).toBe("hold");
|
||||
});
|
||||
|
||||
it("applies the 40px touch-target class on coarse-pointer devices", () => {
|
||||
coarsePointer.value = true;
|
||||
renderButton();
|
||||
const button = screen.getByTestId(BUTTON_TESTID);
|
||||
expect(button.className).toContain("h-10");
|
||||
expect(button.className).toContain("w-10");
|
||||
expect(button.className).not.toContain("h-7");
|
||||
});
|
||||
});
|
||||
|
||||
describe("VoiceInputButton — stored toggle mode is unaffected by pointer kind", () => {
|
||||
it("renders effective toggle and ignores pointer events when stored mode is toggle", () => {
|
||||
voicePrefs.value = { ...voicePrefs.value, mode: "toggle" };
|
||||
coarsePointer.value = false;
|
||||
renderButton();
|
||||
const button = screen.getByTestId(BUTTON_TESTID);
|
||||
expect(button.getAttribute("data-mode")).toBe("toggle");
|
||||
expect(button.getAttribute("data-effective-mode")).toBe("toggle");
|
||||
|
||||
// Pointer handlers must not be attached in toggle mode; pointerdown alone
|
||||
// should not start a recording. Click is what drives toggle.
|
||||
fireEvent.pointerDown(button, { pointerId: 4 });
|
||||
expect(voiceInputResult.start).not.toHaveBeenCalled();
|
||||
fireEvent.click(button);
|
||||
expect(voiceInputResult.start).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
});
|
||||
|
||||
describe("VoiceInputButton — recording-state stop transition", () => {
|
||||
it("invokes stop() on click when state is recording (toggle path)", () => {
|
||||
voicePrefs.value = { ...voicePrefs.value, mode: "toggle" };
|
||||
voiceInputResult.state = "recording";
|
||||
renderButton();
|
||||
const button = screen.getByTestId(BUTTON_TESTID);
|
||||
fireEvent.click(button);
|
||||
expect(voiceInputResult.stop).toHaveBeenCalledTimes(1);
|
||||
expect(voiceInputResult.start).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
|
||||
describe("VoiceInputButton — disabled feature & unsupported fallback", () => {
|
||||
it("renders nothing when voice mode is disabled in settings", () => {
|
||||
voicePrefs.value = { ...voicePrefs.value, enabled: false };
|
||||
const { container } = renderButton();
|
||||
expect(container.firstChild).toBeNull();
|
||||
});
|
||||
|
||||
it("renders the unsupported fallback button when no engine is available", () => {
|
||||
voiceInputResult.supported = false;
|
||||
renderButton();
|
||||
const button = screen.getByTestId(BUTTON_TESTID);
|
||||
expect(button.getAttribute("data-state")).toBe("unsupported");
|
||||
expect(button.getAttribute("aria-label")).toBe("Voice input unavailable");
|
||||
});
|
||||
});
|
||||
@@ -1,6 +1,6 @@
|
||||
"use client";
|
||||
|
||||
import { useCallback, useEffect, useRef } from "react";
|
||||
import { useCallback, useEffect, useRef, useSyncExternalStore } from "react";
|
||||
import { IconLoader2, IconMicrophone, IconPlayerStopFilled } from "@tabler/icons-react";
|
||||
|
||||
import { Button } from "@kandev/ui/button";
|
||||
@@ -68,18 +68,54 @@ function toastForError(toast: ReturnType<typeof useToast>["toast"], err: VoiceEr
|
||||
|
||||
// ── Activation handlers ──────────────────────────────────────────────────
|
||||
|
||||
// Hold mode survives finger drift by claiming the pointer with
|
||||
// setPointerCapture on pointerdown. Without capture, any movement that crosses
|
||||
// the button's bounds (a common occurrence on touch — small finger shift,
|
||||
// soft-keyboard reflow, OS gesture handoff) fires pointerleave and would stop
|
||||
// recording mid-utterance. pointerleave is intentionally NOT wired here for
|
||||
// the same reason. Capture is released on pointerup/pointercancel; both also
|
||||
// trigger stop().
|
||||
function safePointerCapture(target: Element, pointerId: number): void {
|
||||
try {
|
||||
target.setPointerCapture(pointerId);
|
||||
} catch {
|
||||
// setPointerCapture can throw InvalidPointerId on browsers that have
|
||||
// already released the implicit capture (older WebKit). Safe to ignore —
|
||||
// the worst case is the prior behaviour where hold could stop on drift.
|
||||
}
|
||||
}
|
||||
|
||||
function safePointerRelease(target: Element, pointerId: number): void {
|
||||
try {
|
||||
// `hasPointerCapture` is in lib.dom but the runtime check guards against
|
||||
// ancient/headless environments that strip it (some jsdom + happy-dom
|
||||
// versions). If the method is present and reports no capture, skip the
|
||||
// release to avoid the no-op throw Safari occasionally fires.
|
||||
if (typeof target.hasPointerCapture === "function" && !target.hasPointerCapture(pointerId)) {
|
||||
return;
|
||||
}
|
||||
target.releasePointerCapture(pointerId);
|
||||
} catch {
|
||||
// ignore — Safari can throw if capture was already released.
|
||||
}
|
||||
}
|
||||
|
||||
function buildHoldHandlers(start: () => Promise<void>, stop: () => Promise<void>) {
|
||||
return {
|
||||
onPointerDown: (e: React.PointerEvent) => {
|
||||
e.preventDefault();
|
||||
safePointerCapture(e.currentTarget, e.pointerId);
|
||||
void start();
|
||||
},
|
||||
onPointerUp: (e: React.PointerEvent) => {
|
||||
e.preventDefault();
|
||||
safePointerRelease(e.currentTarget, e.pointerId);
|
||||
void stop();
|
||||
},
|
||||
onPointerCancel: (e: React.PointerEvent) => {
|
||||
safePointerRelease(e.currentTarget, e.pointerId);
|
||||
void stop();
|
||||
},
|
||||
onPointerLeave: () => void stop(),
|
||||
onPointerCancel: () => void stop(),
|
||||
};
|
||||
}
|
||||
|
||||
@@ -114,6 +150,34 @@ function useAutoSendOnTranscript(
|
||||
);
|
||||
}
|
||||
|
||||
// Subscribes to `(pointer: coarse)` so the component re-renders when the user
|
||||
// docks/undocks an external pointer (Surface, iPad with Magic Keyboard). SSR
|
||||
// returns false, matching desktop default — mobile-first hydration corrects on
|
||||
// first client paint. Pattern mirrors `useResponsiveBreakpoint`.
|
||||
function subscribeCoarsePointer(callback: () => void): () => void {
|
||||
if (typeof window === "undefined" || typeof window.matchMedia !== "function") return () => {};
|
||||
const mql = window.matchMedia("(pointer: coarse)");
|
||||
mql.addEventListener("change", callback);
|
||||
return () => mql.removeEventListener("change", callback);
|
||||
}
|
||||
|
||||
function getCoarsePointerSnapshot(): boolean {
|
||||
if (typeof window === "undefined" || typeof window.matchMedia !== "function") return false;
|
||||
return window.matchMedia("(pointer: coarse)").matches;
|
||||
}
|
||||
|
||||
function getCoarsePointerServerSnapshot(): boolean {
|
||||
return false;
|
||||
}
|
||||
|
||||
function useIsCoarsePointer(): boolean {
|
||||
return useSyncExternalStore(
|
||||
subscribeCoarsePointer,
|
||||
getCoarsePointerSnapshot,
|
||||
getCoarsePointerServerSnapshot,
|
||||
);
|
||||
}
|
||||
|
||||
function useVoiceShortcut(
|
||||
enabled: boolean,
|
||||
state: VoiceInputState,
|
||||
@@ -192,11 +256,99 @@ export function VoiceInputButton({ onTranscript, onAutoSend, disabled }: VoiceIn
|
||||
);
|
||||
}
|
||||
|
||||
// Hold-to-talk is unreliable on touch even with pointer capture: the platform
|
||||
// reclaims the pointer for system gestures (back swipes, scroll-chains) and the
|
||||
// user's stored preference becomes a trap. Coarse-pointer devices silently get
|
||||
// toggle behaviour; persisted `voiceMode.mode` is left alone so docking a
|
||||
// keyboard restores the user's choice without a save.
|
||||
function resolveEffectiveMode(
|
||||
prefMode: "hold" | "toggle",
|
||||
isCoarsePointer: boolean,
|
||||
): "hold" | "toggle" {
|
||||
return prefMode === "hold" && !isCoarsePointer ? "hold" : "toggle";
|
||||
}
|
||||
|
||||
function resolveTooltip(args: {
|
||||
modelLoad: VoiceModelLoadState;
|
||||
modelLabel: string;
|
||||
state: VoiceInputState;
|
||||
holdMode: boolean;
|
||||
}): string {
|
||||
const { modelLoad, modelLabel, state, holdMode } = args;
|
||||
if (modelLoad.state === "loading") {
|
||||
const pct = Number.isFinite(modelLoad.progress)
|
||||
? Math.min(100, Math.max(0, Math.round(modelLoad.progress * 100)))
|
||||
: 0;
|
||||
return `Downloading ${modelLabel}… ${pct}%`;
|
||||
}
|
||||
return `${TOOLTIP_BY_STATE[state]}${holdMode && state === "idle" ? " (hold)" : ""}`;
|
||||
}
|
||||
|
||||
type VoiceMicButtonProps = {
|
||||
state: VoiceInputState;
|
||||
modelLoad: VoiceModelLoadState;
|
||||
disabled?: boolean;
|
||||
storedMode: "hold" | "toggle";
|
||||
effectiveMode: "hold" | "toggle";
|
||||
isCoarsePointer: boolean;
|
||||
pointerHandlers: ReturnType<typeof buildHoldHandlers> | Record<string, never>;
|
||||
onClick: (() => void) | undefined;
|
||||
};
|
||||
|
||||
// Styled to mirror SubmitButton (h-7 w-7 rounded-full primary fill) so the two
|
||||
// prominent input actions read as a pair on the right of the toolbar.
|
||||
// Recording flips to a destructive fill with a pulsing ring so the active
|
||||
// state is unmistakable even on mobile. Touch-input sizing bumps to 10×10
|
||||
// (40px) for an easier finger target without disturbing the desktop pair.
|
||||
function VoiceMicButton({
|
||||
state,
|
||||
modelLoad,
|
||||
disabled,
|
||||
storedMode,
|
||||
effectiveMode,
|
||||
isCoarsePointer,
|
||||
pointerHandlers,
|
||||
onClick,
|
||||
}: VoiceMicButtonProps) {
|
||||
const isRecording = state === "recording";
|
||||
const isBusy = state === "requesting" || state === "processing" || modelLoad.state === "loading";
|
||||
return (
|
||||
<Button
|
||||
type="button"
|
||||
variant="default"
|
||||
size="icon"
|
||||
aria-label={ARIA_BY_STATE[state]}
|
||||
aria-pressed={isRecording}
|
||||
data-testid="voice-input-button"
|
||||
data-state={state}
|
||||
data-mode={storedMode}
|
||||
data-effective-mode={effectiveMode}
|
||||
disabled={!!disabled || (isBusy && state !== "recording")}
|
||||
onClick={onClick}
|
||||
{...pointerHandlers}
|
||||
className={cn(
|
||||
"rounded-full cursor-pointer relative select-none",
|
||||
isCoarsePointer ? "h-10 w-10" : "h-7 w-7",
|
||||
isRecording && "bg-destructive text-destructive-foreground hover:bg-destructive/90",
|
||||
)}
|
||||
>
|
||||
<ButtonIcon state={state} modelLoad={modelLoad} />
|
||||
{isRecording && (
|
||||
<span
|
||||
aria-hidden
|
||||
className="absolute inset-0 rounded-full ring-2 ring-destructive/40 animate-pulse"
|
||||
/>
|
||||
)}
|
||||
</Button>
|
||||
);
|
||||
}
|
||||
|
||||
function EnabledVoiceInputButton({ onTranscript, onAutoSend, disabled }: VoiceInputButtonProps) {
|
||||
const { toast } = useToast();
|
||||
const voiceMode = useAppStore((s) => s.userSettings.voiceMode);
|
||||
const handleError = useCallback((err: VoiceError) => toastForError(toast, err), [toast]);
|
||||
const wrappedTranscript = useAutoSendOnTranscript(onTranscript, onAutoSend, voiceMode.autoSend);
|
||||
const isCoarsePointer = useIsCoarsePointer();
|
||||
|
||||
const { supported, state, modelLoad, start, stop, cancel } = useVoiceInput({
|
||||
onTranscript: wrappedTranscript,
|
||||
@@ -217,19 +369,13 @@ function EnabledVoiceInputButton({ onTranscript, onAutoSend, disabled }: VoiceIn
|
||||
// the button silently left mobile users with no discoverable feedback.
|
||||
if (!supported) return <UnsupportedVoiceButton disabled={disabled} />;
|
||||
|
||||
const isRecording = state === "recording";
|
||||
const isBusy = state === "requesting" || state === "processing" || modelLoad.state === "loading";
|
||||
const holdMode = voiceMode.mode === "hold";
|
||||
|
||||
const effectiveMode = resolveEffectiveMode(voiceMode.mode, isCoarsePointer);
|
||||
const holdMode = effectiveMode === "hold";
|
||||
const pointerHandlers = holdMode ? buildHoldHandlers(start, stop) : {};
|
||||
const onClick = holdMode ? undefined : buildToggleHandler(state, start, stop);
|
||||
|
||||
const modelLabel = whisperModelConfig(voiceMode.whisperWebModel).label;
|
||||
const tooltipText = resolveTooltip({ modelLoad, modelLabel, state, holdMode });
|
||||
|
||||
// Styled to mirror SubmitButton (h-7 w-7 rounded-full primary fill) so the
|
||||
// two prominent input actions read as a pair on the right of the toolbar.
|
||||
// Recording flips to a destructive fill with a pulsing ring so the active
|
||||
// state is unmistakable even on mobile.
|
||||
return (
|
||||
<div className="flex items-center gap-1.5">
|
||||
<VoiceModelLoadIndicator
|
||||
@@ -239,37 +385,18 @@ function EnabledVoiceInputButton({ onTranscript, onAutoSend, disabled }: VoiceIn
|
||||
/>
|
||||
<Tooltip>
|
||||
<TooltipTrigger asChild>
|
||||
<Button
|
||||
type="button"
|
||||
variant="default"
|
||||
size="icon"
|
||||
aria-label={ARIA_BY_STATE[state]}
|
||||
aria-pressed={isRecording}
|
||||
data-testid="voice-input-button"
|
||||
data-state={state}
|
||||
data-mode={voiceMode.mode}
|
||||
disabled={!!disabled || (isBusy && state !== "recording")}
|
||||
<VoiceMicButton
|
||||
state={state}
|
||||
modelLoad={modelLoad}
|
||||
disabled={disabled}
|
||||
storedMode={voiceMode.mode}
|
||||
effectiveMode={effectiveMode}
|
||||
isCoarsePointer={isCoarsePointer}
|
||||
pointerHandlers={pointerHandlers}
|
||||
onClick={onClick}
|
||||
{...pointerHandlers}
|
||||
className={cn(
|
||||
"h-7 w-7 rounded-full cursor-pointer relative select-none",
|
||||
isRecording && "bg-destructive text-destructive-foreground hover:bg-destructive/90",
|
||||
)}
|
||||
>
|
||||
<ButtonIcon state={state} modelLoad={modelLoad} />
|
||||
{isRecording && (
|
||||
<span
|
||||
aria-hidden
|
||||
className="absolute inset-0 rounded-full ring-2 ring-destructive/40 animate-pulse"
|
||||
/>
|
||||
)}
|
||||
</Button>
|
||||
/>
|
||||
</TooltipTrigger>
|
||||
<TooltipContent>
|
||||
{modelLoad.state === "loading"
|
||||
? `Downloading ${modelLabel}… ${Number.isFinite(modelLoad.progress) ? Math.min(100, Math.max(0, Math.round(modelLoad.progress * 100))) : 0}%`
|
||||
: `${TOOLTIP_BY_STATE[state]}${holdMode && state === "idle" ? " (hold)" : ""}`}
|
||||
</TooltipContent>
|
||||
<TooltipContent>{tooltipText}</TooltipContent>
|
||||
</Tooltip>
|
||||
</div>
|
||||
);
|
||||
|
||||
@@ -7,6 +7,7 @@ import type {
|
||||
TaskSessionState,
|
||||
} from "../../lib/types/http";
|
||||
import type { Agent, AgentProfile } from "../../lib/types/http-agents";
|
||||
import type { VoiceModeSettings } from "../../lib/types/http-voice";
|
||||
import type {
|
||||
SSHAgentReadinessResponse,
|
||||
SSHProbeShellsResponse,
|
||||
@@ -609,6 +610,7 @@ export class ApiClient {
|
||||
default_utility_model?: string;
|
||||
sidebar_views?: unknown[];
|
||||
kanban_view_mode?: string;
|
||||
voice_mode?: VoiceModeSettings;
|
||||
}): Promise<void> {
|
||||
await this.request("PATCH", "/api/v1/user/settings", settings);
|
||||
}
|
||||
|
||||
@@ -0,0 +1,90 @@
|
||||
import { test, expect } from "../../fixtures/test-base";
|
||||
import type { SeedData } from "../../fixtures/test-base";
|
||||
import type { ApiClient } from "../../helpers/api-client";
|
||||
import { SessionPage } from "../../pages/session-page";
|
||||
|
||||
// Mobile (Pixel 5) regression coverage for the voice-mode entry point and
|
||||
// the chat composer's mic button. Filename matches /mobile-.*\.spec\.ts/ so
|
||||
// the `mobile-chrome` Playwright project picks it up.
|
||||
//
|
||||
// Two assertions:
|
||||
// 1. The Voice Mode settings page renders all cards at mobile width when
|
||||
// navigated to directly — guards against the "setting missing" symptom.
|
||||
// 2. The mic button mounts on the mobile chat composer with the coarse-pointer
|
||||
// override: a user with stored `voiceMode.mode = "hold"` gets toggle
|
||||
// behaviour (data-effective-mode = "toggle") and a 40px touch target.
|
||||
|
||||
async function seedTask(apiClient: ApiClient, seedData: SeedData, title: string) {
|
||||
return apiClient.createTaskWithAgent(seedData.workspaceId, title, seedData.agentProfileId, {
|
||||
description: "/e2e:simple-message",
|
||||
workflow_id: seedData.workflowId,
|
||||
workflow_step_id: seedData.startStepId,
|
||||
repository_ids: [seedData.repositoryId],
|
||||
});
|
||||
}
|
||||
|
||||
test.describe("Mobile voice mode", () => {
|
||||
test.describe.configure({ retries: 1, timeout: 60_000 });
|
||||
|
||||
test("Voice Mode settings page renders every card at mobile width", async ({ testPage }) => {
|
||||
await testPage.goto("/settings/voice-mode");
|
||||
|
||||
// Page header confirms route resolved to the right shell. The page title
|
||||
// is rendered by SettingsSection — match the heading text exactly so we
|
||||
// don't pick up the sidebar entry that shares the label.
|
||||
await expect(testPage.getByRole("heading", { name: "Voice Mode", exact: true })).toBeVisible({
|
||||
timeout: 15_000,
|
||||
});
|
||||
|
||||
// The five cards rendered by `voice-mode-settings.tsx` at mobile width.
|
||||
await expect(testPage.getByText("Enable Voice Input")).toBeVisible();
|
||||
await expect(testPage.getByText("Transcription Engine")).toBeVisible();
|
||||
await expect(testPage.getByText("Behavior")).toBeVisible();
|
||||
await expect(testPage.getByText("Whisper Web Model")).toBeVisible();
|
||||
await expect(testPage.getByText(/Shortcut$/)).toBeVisible();
|
||||
|
||||
// The mobile sidebar trigger (md:hidden) lives in the topbar; assert it
|
||||
// exists so we know the sidebar entry chain that exposed this page is
|
||||
// not silently removed by a future layout change.
|
||||
const sidebarTrigger = testPage.locator('button[data-sidebar="trigger"]');
|
||||
await expect(sidebarTrigger).toBeVisible();
|
||||
});
|
||||
|
||||
test("voice input mic button mounts on the mobile chat composer with the coarse-pointer touch target and effective toggle mode", async ({
|
||||
testPage,
|
||||
apiClient,
|
||||
seedData,
|
||||
}) => {
|
||||
// Seed hold mode so the coarse-pointer override assertion is non-vacuous:
|
||||
// data-mode="hold" (stored pref) vs data-effective-mode="toggle" (override)
|
||||
// proves the coarse-pointer branch fired, not that the default was toggle.
|
||||
await apiClient.saveUserSettings({
|
||||
voice_mode: {
|
||||
enabled: true,
|
||||
engine: "auto",
|
||||
language: "auto",
|
||||
mode: "hold",
|
||||
auto_send: false,
|
||||
whisper_web_model: "base",
|
||||
},
|
||||
});
|
||||
|
||||
const task = await seedTask(apiClient, seedData, "Mobile voice button");
|
||||
await testPage.goto(`/t/${task.id}`);
|
||||
const session = new SessionPage(testPage);
|
||||
await session.waitForLoad();
|
||||
|
||||
const micButton = testPage.getByTestId("voice-input-button");
|
||||
await expect(micButton).toBeVisible({ timeout: 15_000 });
|
||||
|
||||
// Pixel 5 advertises `(pointer: coarse)`. Three signals that the coarse-
|
||||
// pointer override fired:
|
||||
// - data-mode="hold" (the stored user preference)
|
||||
// - data-effective-mode="toggle" (the override: hold→toggle on coarse)
|
||||
// - h-10 w-10 (40px touch target, only set when isCoarsePointer=true)
|
||||
await expect(micButton).toHaveAttribute("data-mode", "hold");
|
||||
await expect(micButton).toHaveAttribute("data-effective-mode", "toggle");
|
||||
await expect(micButton).toHaveClass(/(^|\s)h-10(\s|$)/);
|
||||
await expect(micButton).toHaveClass(/(^|\s)w-10(\s|$)/);
|
||||
});
|
||||
});
|
||||
Reference in New Issue
Block a user