When I moved several long-form writing workflows entirely offline, the single biggest friction point was reliable, accurate dictation that respected privacy and worked without an internet connection. Cloud ASR (automatic speech recognition) is great for accuracy, but for sensitive notes, interviews, or fieldwork where connectivity is spotty, on-device speech recognition is the only realistic option. I spent months evaluating and integrating different engines into my dictation pipeline, and in this article I'll walk through the tradeoffs I ran into: accuracy vs. latency, resource usage, language coverage, ease of integration, and licensing limits.
What I needed from an offline dictation engine
Before comparing tools, it's useful to be explicit about the requirements that drove my choices:
High word‑level accuracy for clear, conversational speech (medical and legal jargon occasionally)Low real‑time latency for live dictationModest memory and CPU demands to run on modern laptops and some ARM devicesSupport for multiple languages and local customization (custom vocabularies or language models)Permissive licensing suitable for a commercial product or research useReasonable integration options (Python bindings or portable C/C++ libraries)With those constraints, I focused on five families of engines: Whisper (local builds), Vosk, Kaldi-derived systems, Mozilla DeepSpeech (and forks), and newer transformer-based models like Wav2Vec 2.0 running on ONNX/Torch. I also experimented with lightweight options like PocketSphinx and optimized ports like whisper.cpp.
Short descriptions of the contenders
Whisper (local) — OpenAI's Whisper models are transformer-based and surprisingly tolerant of noisy audio and diverse accents. While the full models are large, there are smaller checkpoints and community projects (whisper.cpp, GGML builds) that make on-device use practical.Vosk — A wrapper around Kaldi with prebuilt models for many languages. Vosk is easy to integrate (Python, Java, C#), supports incremental recognition (useful for streaming dictation), and has small-ish models for embedded use.Kaldi — The research-grade toolbox. Extremely flexible and can be tuned for specific domains, but building, training, and packaging models is heavyweight. Best when you need bespoke acoustic/language models.Mozilla DeepSpeech (and forks like Coqui) — RNN-based systems that were very popular for on-device work. Coqui continues the project and offers reasonably lightweight models and straightforward APIs.Wav2Vec 2.0 / ONNX — Transformer-based acoustic models that can be fine-tuned for speech recognition. When exported to ONNX and paired with a lightweight decoder, they can run on device with good accuracy.PocketSphinx — Ultra-lightweight, designed for embedded devices and keyword spotting. Not great for full dictation accuracy but useful for low-power scenarios.Head-to-head comparison table (practical metrics)
| Engine | Typical device | Accuracy (clean speech) | Latency | Memory/CPU | Languages | Notes |
|---|
| Whisper (small/medium) | Laptop, ARM with optimizations | High | Medium (real-time with small models) | Moderate–High | Many | Best noise robustness; large models heavy |
| Vosk (Kaldi) | Laptop, Raspberry Pi | Good | Low | Low–Moderate | Many | Streaming-ready; easy vocab tweaks |
| Kaldi (custom) | Server or powerful laptop | Very high (with training) | Low–Medium | High | Any (requires data) | Best for custom domains; complexity is high |
| Coqui / DeepSpeech | Laptop, mobile | Good | Low | Low–Moderate | Several | Simple API; model quality varies |
| Wav2Vec 2.0 (on device) | Laptop, GPU/CPU | High | Depends (optimized good) | Moderate–High | Depends on fine-tune | Needs fine‑tuning and decoding stack |
| PocketSphinx | Embedded devices | Poor–Fair | Very low | Very low | Some | Great for keywords, not full dictation |
Practical observations from my tests
I ran the same 30-minute interview recording through each engine and used a mix of live dictation sessions. Here are the things that mattered most in daily use.
Noise robustness — Whisper and Wav2Vec models handled background noise and overlapping speech much better than older RNN-based engines. If you often dictate in cafés or on transit, that makes a huge difference in accuracy and editing time.Latency for live dictation — Vosk and Coqui give the snappy, sub-second word streaming that feels like a live transcription assistant. Whisper small models can be near real-time with optimized builds (whisper.cpp), but large models introduce buffer delays.Resource usage and battery life — Lighter Kaldi-based models and DeepSpeech/Coqui are kinder to CPU and battery. Transformer models (Whisper, Wav2Vec) are heavier unless you run them with quantized weights or on-device NN accelerators.Language coverage and accents — Vosk offers many community models; Whisper has strong multilingual performance. If you need obscure languages, Kaldi/Vosk with community models is often the fastest route.Customization — Kaldi wins if you want to train a domain-specific model (e.g., medical transcription vocabulary). Vosk exposes ways to add custom words and phrase lists that helped with proper nouns in my workflows.Integration and deployment — Vosk was plug-and-play for Python apps. Whisper has a great Python ecosystem but requires care for performance; whisper.cpp and ggml make native integration simpler on resource‑constrained devices.Workflow patterns I ended up using
I ended up mixing engines depending on context:
For interview transcription where accuracy mattered and I could afford post-processing time, I ran recordings through Whisper (medium) on my laptop overnight. The noise robustness and punctuation handling reduced editing time.For live dictation during note-taking, I used Vosk with a custom vocabulary and real-time streaming. The latency was minimal and I could correct on the fly.For embedded, offline field devices (Raspberry Pi), I used a quantized Vosk model or whisper.cpp tiny model. PocketSphinx was only useful for short keyword commands in constrained hardware.Licensing and commercial considerations
Licensing matters if you’re building a product. Whisper code and model weights were released with permissive terms, but be mindful of updates and community forks. Vosk models are often Apache/MIT-friendly, Kaldi is research-focused with flexible terms, and Coqui (DeepSpeech) has a permissive stance. Always verify the exact license of the model you plan to redistribute—some community models embed third‑party data that can carry constraints.
Tips for improving on-device dictation accuracy
Use a decent microphone and noise profile—audio quality beats tiny model tweaks.Apply basic audio preprocessing: simple high-pass filters, normalization, and aggressive VAD (voice activity detection) reduce junk input.Use custom vocabularies or biasing lists for names, jargon and technical terms.Quantize models to reduce memory and latency, but validate accuracy impact.Consider a hybrid approach: local streaming for low-latency editing, batch offline runs through a heavier model for final transcripts.If you want, I can share the exact scripts and model checkpoints I used for each engine, along with Dockerfiles and whisper.cpp build flags that made Whisper usable on my M1 and a Raspberry Pi 4. Tell me what device and language you care about most and I’ll tailor the recommendations and code snippets.
