The topic I trusted my local LLM with everything except my code, and I had it exactly backward is currently the subject of lively discussion — readers and analysts are keeping a close eye on developments.
This is taking place in a dynamic environment: companies’ decisions and competitors’ reactions can quickly change the picture.
I run enough local inference hardware that my electricity provider sends me a Christmas card, so when I decided to spend a week routing my day through a local LLM, I thought I knew exactly where to draw the line. Local models would handle the everyday stuff — summaries, research triage, drafting, the endless “what does this error mean” questions — and coding would stay on the frontier models, because coding is the hard part. That’s been my philosophy for over a year: local for the simple stuff, cloud for anything that matters.
One week later, I’d gotten it exactly backward. The everyday tasks were where the local model quietly disappointed me, and the one workload I’d fenced off, the code, turned out to be the thing it was genuinely built for. I didn’t need a full week to see what was going wrong, and the right way to split the tasks was the opposite of what I was building.
My smart home camera alerts used to be useless, but now they tell me what actually happened
The setup was simple enough. Run Qwen3-Coder-30B-A3B on the Strix Halo box through Lemonade Server, reachable over the network, and have my RTX 5090 machine on standby for bigger contexts. Everything I’d normally throw at a cloud chatbot went local instead: article research, log file diving, summarizing changelogs, brainstorming article ideas, the lot. Coding stayed on Claude and Codex, with Zed doing a little bit of simple edits because I’d internalized the idea that code is where the model quality mattered most, and I’d seen enough half-hallucinated shell scripts to be wary.
That caution wasn’t from general distrust of AI. Frontier models dominate the agentic side of coding, where multi-file refactors, ambiguous requirements, zero-shot prompts, and recovering from failed tool calls are all objectively better. My mistake was assuming that gap applied to all coding and that every day non-coding tasks were the easy tier. They’re not, but they are hard in a different way.
For the first few days, the experiment looked successful. The local model was able to summarize websites or GitHub repos for me and give me installation steps competently. It could answer factual questions well enough, never hit a rate limit, and never asked me to upgrade to the next tier or add API credits. And it worked when the internet was down, which happened once due to a thunderstorm.
But “fine” is never really fine, and the cost of use shifted from a subscription to my time. Everything I offloaded to the local model was hard to verify. I started reading the source material for summaries as a test and continued reading them because I couldn’t always trust the output. When I asked for research triage, it either missed easily found materials or confidently cited things that didn’t exist.
for example, I got it to summarize OpenClaw for me and tell me how to install it. The results were usable, although very surface-level. Then I asked to research if any users have reported issues with the software, because we know OpenClaw has plenty of issues. The local LLM confidently said that it couldn’t find GitHub issues, user forum issues, error messages, or troubleshooting documentation, and that it seems to “have a very positive reception with users.”
If I’d taken that at face value, I’d be in trouble, with secrets spilling out of every port. I asked Claude the same thing and got an installation process with multiple layers of security hardening before adding anything else to OpenClaw, and guidance on how to run it in a sandboxed VM for security. It could have turned into a very expensive mistake. As it turned out, the expense was mostly time, because I had to read the sources myself anyway.
I can now spin up new containers with simple conversational prompts
I learned the hard way that I’d been running things backward when a late-night coding session hit my token limits. I didn’t want to wait for a reset, so I pointed the session at the local LLM I’d been keeping out of my editor. I expected to have to babysit it, or at least answer more questions than the cloud model usually asks. But it found the bug and fixed it, without me having to do anything other than approve some tool calling.
And when it did fail, it was a loud failure, with tests failing, instead of plausible-sounding nonsense that I had to go fact-check. That’s the difference: I hadn’t shown that local models are better at code than prose, just that code is the one domain I don’t have to trust the output. The toolchain handles the trust for me.
Local models are still clearly worse at the hard end of coding. Ask one to decompose an ambiguous feature request across a dozen files, and it’ll wander off, fumble a tool call, or get stuck in a loop in ways that Claude or Codex’s models rarely do. And there’s an associated time cost to using local models, because the review-and-correction loop takes significantly longer than with frontier models.
I’m not pretending that local models are a replacement for cloud models; most agentic-first workflows are designed around Claude Code or similar agent harnesses, and even the smallest cloud models outperform local models. But it depends on what your coding day consists of. Mine is mostly small stuff, scripts to rename screenshots or benchmark runs, regex that I refuse to learn for how often I use it, unit tests for something I already wrote, and digging through logfiles to find out why something isn’t running as expected.
That’s most of my coding hours, single-file, self-verifying, and uncomplicated. The larger stuff stays on Claude, where the tokens are put to good use, but for the small stuff, local has been a decent enough replacement.
I went into this week thinking I was protecting my code from a weaker model. I came out realizing I’d been protecting the wrong asset the whole time. The local LLM now lives in my editor, grinding through scripts and tests where the compiler keeps it honest, and the cloud models handle everything else, where a confident mistake can actually hurt me. If you’re running local models and wondering where they fit, don’t start with a mental org chart of “easy tasks” and “hard tasks”. Start with which failures you can catch for free.