AI

When I moved several long-form writing workflows entirely offline, the single biggest friction point was reliable, accurate dictation that respected privacy and worked without an internet connection. Cloud ASR (automatic speech recognition) is great for accuracy, but for sensitive notes, interviews, or fieldwork where connectivity is spotty, on-device speech recognition is the only realistic option. I spent months evaluating and integrating...

Read more...

I remember the first time I tried to run a modern language model on my laptop: it was slow, memory-starved, and I spent more time swapping RAM than actually getting useful responses. Since then I’ve tested pruning, quantization, on-device runtimes and — most importantly — model distillation. Distillation is the technique that finally let me run capable models locally without paying cloud fees or sacrificing privacy. In this piece I’ll...

Read more...