I went local to stop letting a company quietly decide things for me. My third command was ollama run. And just like that, something was quietly deciding everything again.
Every post in this series was, underneath, a decision someone makes. Which quantization to download. How many of the model's layers ride on the GPU. The size and type of your KV cache. Whether speculative decoding is even on the table. Whether your requests get batched with anyone else's. On the API, a company made every one of those calls and never showed you the menu.
Locally, you'd think the menu is finally yours. It isn't — not by default. The moment you type ollama run, Ollama makes all of those choices for you. It picks the quant, sets the layer split, sizes the cache, chooses the engine, and tucks every lever out of sight. It isn't really a tool. It's a provider you happen to host on your own desk.
That's a gift on day one and a cage by day thirty. So here's the decision you didn't know you were making.
These runtimes aren't rivals over the same job, they're different layers of the stack. Ollama is an experience layer. Under it sit the real engines, llama.cpp and MLX. Off to the side, vLLM is a serving system. "Which runtime" is really "which layer do I want my hands on."
Ollama is the easy button: it decides everything, asks nothing, runs in one line. The right place for almost everyone to start, mine included. Drop a level and you reach llama.cpp, the engine living under Ollama. It runs on nearly any hardware and gives back every knob Ollama hid, the place you'd actually switch on a draft model or pin your KV cache to a chosen precision. On a Mac, the fast engine is MLX, Apple's own, built for your unified memory, which is exactly why modern Ollama reaches for it under the hood. And vLLM is the odd one out, built for the crowd: it batches many people against one model, which makes it unbeatable for a server and close to pointless for solo you.
See the shape of it? You never stopped choosing a provider. You moved the choice from a company to a runtime. And the runtime you pick decides which of these hard-won tradeoffs you actually get to touch. Choosing Ollama isn't sidestepping the decision. It's choosing the option that makes the decision for you.
Name your runtime and what it optimizes for, out loud. If you just want a model that works, stay on Ollama and let it drive; that's not a compromise, it's the correct default. But the day you reach for a knob this series taught you and Ollama won't show it to you, you've outgrown the easy button. The engine underneath, llama.cpp or MLX, is where the levers live. And if you ever serve more than yourself, that's a vLLM job, not an Ollama one.
This series opened on everything the API was quietly doing for you. It closes on the part nobody mentions: going local doesn't end the quiet deciding. It just hands you the keys and lets you choose how much to look.
Subscribe and I'll send each one as it lands.
You didn't take the model off the grid. You became the grid.