How to set up a sub‑50ms private multimodal assistant on an intel nuc using rust and onnxruntime

I recently spent a weekend auditing a couple of consumer VPN setups—one based on WireGuard, one using the ProtonVPN client—to answer the question I get a lot: "How can I check a VPN for leaks and hidden telemetry without becoming a reverse‑engineering expert?" I wanted something practical you can reproduce with open tools and a modest amount of time. Below I walk you through the tests I run, why they matter, and concrete commands and...

Read more...

I spent the past two weeks running a practical, hands‑on comparison of inexpensive USB‑C docks to answer a simple but frustrating question: which budget docks keep a MacBook Pro happily charging and an external display stable when the laptop is put under sustained GPU load? It’s a real‑world problem — you open Final Cut, DaVinci Resolve, or a heavy browser session with many WebGL tabs and suddenly the external monitor starts...

Read more...

I want to share a practical, hands‑on approach I use to detect supply‑chain tampering in npm packages before anything reaches CI deploys. Over the years I’ve combined small, free tools and reproducible‑build practices to make a fast, local gate that catches the most common and stealthiest tricks attackers use — malicious postinstall scripts, rogue tarballs, or silently altered published code. Below I walk through the mindset, the...

Read more...

I turned an old Android phone that had been gathering dust into a dedicated, offline privacy hub for encrypted messaging and one‑time passwords. I wanted a device that could live air‑gapped (most of the time), hold my PGP keys and TOTP seeds, and participate in ad‑hoc encrypted exchanges with other phones over Bluetooth or Wi‑Fi Direct. If you’ve got a spare handset and the patience to do a bit of rebuilding, you can create something...

Read more...

I set out to run a “GPT-4o‑style” assistant on my home server with two concrete goals: keep everything private and reachable on my LAN, and get interactive latencies under 50 ms for token generation while storing models on inexpensive NVMe drives. After testing several toolchains, models and hardware configurations, I ended up with a reproducible approach that balances cost, latency and real-world usefulness. Below I walk through what...

Read more...

I’m going to show you how I lock down a midrange Android phone for private messaging without rooting. The goal is practical: make it hard for casual snoops, resilient against common app-level leaks, and preserve strong cryptographic privacy for messages—without tinkering with the bootloader or installing a custom ROM. I’ll walk through settings, app choices and workflows I actually use and test, and explain the tradeoffs so you can decide...

Read more...

When customer support teams adopt retrieval-augmented generation (RAG) to power chatbots, the promise is compelling: fast, contextually-aware answers grounded in a company's own documentation. In practice, however, one problem keeps surfacing — hallucinations. These are fluent, plausible-sounding responses that confidently state incorrect facts or invent citations. I've worked with product and security teams who’ve felt that a seemingly...

Read more...

When I started evaluating self-hosted vector databases for on-device LLM search, I expected a straightforward tradeoff: pick the fastest engine and you're done. Reality was messier. The right choice depends on workload patterns, hardware constraints, embedding strategy, and how much operational complexity you’re willing to accept. Below I walk through what I learned comparing Milvus, pgvector and Chroma—practical differences, deployment...

Read more...

I recently spent a week building a cheap, repeatable workflow to detect malicious firmware implants on consumer routers using nothing more than a Raspberry Pi and free tools. If you manage home networks, small office gear, or test client devices, this is a practical way to get visibility into whether a router’s firmware has been tampered with without buying an expensive hardware analyzer. Below I walk through the signals I look for, the tools...

Read more...

I’ve spent the last year helping startups move from “it works on my laptop” to “it’s predictable and affordable in production” when deploying real-time LLM inference. One recurring headache is cloud costs that explode unpredictably because inference usage is measured in tokens, not requests—and tokens vary wildly. In this guide I’ll walk through how I measure token-level costs, build token-aware autoscaling, and put practical...

Read more...