How a spoken request becomes a Windows action
Fluent separates speech recognition from action planning. A constrained Windows listener waits for the Fluent wake phrase. A microphone click or the Ctrl+Alt+, shortcut can also arm one utterance. Chromium captures that command, and a bundled whisper.cpp runtime transcribes it locally.
The text request then enters the same command path as typed input. A hosted planning provider selects from bounded automation tools, such as observing the accessibility tree, opening an app, clicking a named control, or typing text. The app displays action progress and accepts cancellation while work is interruptible.
- Wake phrase, microphone click, or push-to-talk arms one command.
- Raw speech audio is transcribed on the device, not by a cloud speech API.
- The text request is sent to the configured hosted planning provider.
- The planner uses Fluent tools backed by Windows UI Automation and PowerShell.
- The overlay shows progress and lets the user cancel an active run.
Natural requests are different from command memorization
Traditional voice control often asks the user to remember an exact command, identify a control by name, or select a numbered overlay. Fluent explores a different layer: the user describes the outcome and the planner decomposes it into smaller semantic actions.
That approach can reduce command syntax, but it also introduces model uncertainty. Fluent therefore treats visibility, cancellation, semantic targets, and consequence-aware approval as core requirements, not optional polish.
Where Fluent may fit
Fluent is being designed for people who want to reduce repeated keyboard and mouse work across changing Windows applications. That may include people navigating pain, fatigue, tremor, injury, limited reach, or simply a workflow where speaking the outcome is easier than relaying every click.
Voice is not the only path. Typed requests use the same surface when speech is private, tiring, unreliable, or unavailable. Optional gaze can add a semantic target when a phrase such as "open that" needs visible context.
- Cross-app tasks where the desired outcome is easier to say than the exact route.
- Workflows that benefit from a visible action trace and a nearby cancel path.
- Users who need voice and text to coexist rather than compete as separate modes.
- Early testers who can tolerate preview setup and help document failures.
What Fluent does not promise yet
Fluent is not a replacement for every established accessibility tool. It currently requires Windows, a supported hosted planning-provider API key, and developer-oriented setup. The published product record identifies missing first-run provider setup, incomplete installed-app golden paths, gaze recovery gaps, and safety paths that still require hardening.
A production claim must eventually be supported by fresh-install success, repeatable workflow results, consequence fixtures, accessible recovery, signed distribution, and user testing. Until those gates pass, Fluent should be evaluated as a transparent preview.