Local LLM vs. cloud API: what it costs and when each one wins

The real question is who owns the GPU

"Local or cloud?" sounds like a binary choice. It isn't. The decision is really about who owns and operates the hardware that runs the model, and there are five common answers:

1. Your laptop or desktop. A small model running on the machine you already have.
2. A machine you buy for the job. A PC with a strong GPU, or a Mac with a lot of memory.
3. A GPU you rent by the hour. Still your own deployment, on someone else's hardware.
4. A hosted open-model API. Someone else runs an open model and charges per token.
5. A frontier API. Claude, GPT, Gemini and similar, billed per token.

Options 1 and 2 are "local" in the strict sense. Option 3 is a middle ground. Options 4 and 5 are "cloud". This article compares them on privacy, quality, cost and effort, with real prices, so you can decide where on that spectrum your workload belongs.

What "local" actually means

One misconception comes up more than any other: you cannot run the flagship Claude, GPT or Gemini models locally. They are only available through their providers' APIs and apps. What you can run are open-weight models, whose trained parameters are published so anyone can download them. Examples include Llama, Qwen, DeepSeek, Mistral, Gemma and OpenAI's gpt-oss. Many of them are very good, but they are different models from the ones in the cloud.

Running one locally usually means one of three things:

• A desktop app such as Ollama or LM Studio. You download a model and chat with it in minutes.
• An inference engine such as llama.cpp, which gives you more control and runs on modest hardware.
• A serving stack such as vLLM on a GPU server, built to handle many users at once.

Our Run Your LLM directory compares these tools. Step-by-step setup guides are coming in separate posts, so this article stays focused on the decision.

The case for running locally

• Your data stays on your machine. Prompts and documents never leave the device. For client files, internal code, medical or legal text, or anything under an NDA, that can settle the question on its own. It also simplifies GDPR questions, because no personal data is passed to an outside processor.
• No cost per token. Once the hardware exists, the next million tokens cost you a little electricity. If you already own a capable machine, your marginal cost is close to zero.
• It works offline and never gets rate-limited. No outages, no quota errors, no waiting in a queue during a busy hour.
• The model never changes underneath you. Cloud providers retire and update models. A local model file behaves the same way next year as it does today, which matters for repeatable results.
• You can customize it. You can fine-tune an open model on your own data, change its behavior, or run versions without a provider's content filters.
• Low latency for small models. There is no network round trip, so short responses from a small model can feel instant.

The case for the cloud

• Higher quality where it counts. The best closed models still lead on hard reasoning, long multi-step coding tasks and tricky instructions. The models that fit on consumer hardware are a tier below even the largest open ones.
• No operations work. Scaling, uptime, security patches and model updates are someone else's job. You write a few lines of code and call an endpoint.
• Cheap at low and moderate volume. You pay only for what you use. As the numbers below show, that is often less than the monthly cost of owning hardware.
• Features you can't easily replicate. Very large context windows, image and audio input, built-in tools, prompt caching, and 50% batch discounts at several providers.
• Speed at scale. Hosted inference often produces tokens faster than a consumer GPU, and it handles many requests in parallel.
• Easy to switch. If a better or cheaper model appears next month, you change a model name. Hardware can't be changed that quickly.

What it actually costs

This is where most comparisons stay vague, so here are real numbers. We'll use one example workload throughout: a small app that handles 1,000 requests a day, each with about 1,500 input tokens and 500 output tokens. That adds up to 45 million input tokens and 15 million output tokens a month, or 60 million tokens in total.

Cloud: the monthly bill

Standard-tier prices per million tokens from the calculator. * Approximate third-party price. Anthropic's batch tier halves the Claude prices when you can wait for results. Prices change; check the calculator for current figures.

The same workload costs anywhere from under $4 to $480 a month depending on the model. The tier you choose matters far more than whether the model is local or hosted. That range is the first thing to keep in mind when you weigh a hardware purchase against an API bill. For why output tokens dominate these totals, see why output tokens cost more than input tokens.

Local: what hardware costs

Approximate 2026 street prices in USD. Hardware and memory prices move quickly and vary by region and condition. "Monthly, all-in" is the purchase price spread over 3 years plus electricity of about $10–15 a month for a GPU PC used 4 hours a day. Rented-GPU rates are about $0.40–1.00 per hour for a 24 GB card and $2–4 per hour for an 80 GB card, 730 hours a month.

Where the break-even sits

Take the "new PC with a good GPU" row at roughly $80 a month all-in. How many tokens a month must you process before that beats paying per token? Using a blend of three input tokens to one output token:

Break-even = $80 ÷ blended price per million tokens. Blended = 75% input rate + 25% output rate.

Read this table carefully, because it cuts both ways. Against the expensive models, local looks like a bargain at just 10–20 million tokens a month. But a 30B-class model on a home GPU is not a substitute for Opus or Sonnet. The fair comparison is against the cheap tiers, and there the break-even is hundreds of millions of tokens a month. A single consumer GPU generating nonstop produces on the order of 100 million tokens a month at best, and far less in real use.

Our example workload of 60 million tokens a month would cost about $12 on Haiku 5.5 or $10.50 on a hosted Llama. Owning a machine to save that is a poor trade. Pure cost savings from going local are rare for individuals and small teams. The real reasons to go local are the ones in the earlier section, such as privacy, offline use and control, or the fact that you already own the hardware.

The costs people forget

• Your time. Installing, choosing a model, tuning settings, updating and fixing things all take hours. At any reasonable hourly rate, that dwarfs a small API bill.
• Idle hardware. A GPU that sits unused still costs money. Cloud billing stops when your usage does.
• Quality-adjusted cost. A cheaper model that needs two attempts, or produces answers you must double-check, isn't cheaper.
• Scaling. One machine serves a handful of users. Ten times the traffic means ten times the hardware, or queues.

Can you run a solid LLM locally for free?

Yes, with an honest definition of "free." The software costs nothing: Ollama, LM Studio and llama.cpp are free to download, and many strong models are free to use. What isn't free is the machine and the electricity it uses. If you already own a reasonably recent computer, you can start for $0.

What matters most is memory. A model compressed to 4-bit precision (a technique called quantization, which shrinks a model with a small loss in quality) needs roughly 0.6 GB per billion parameters, plus a few gigabytes for the conversation. That gives these realistic tiers:

Memory means VRAM on a PC GPU, or unified memory on an Apple Silicon Mac. A PC without enough VRAM can spill into system RAM, but generation becomes much slower.

Be realistic about quality. Local models in the 8–30B range are excellent for drafting, summarizing, rewriting, extraction, classification and simple coding help. They struggle more with complex multi-step reasoning, large codebases and subtle instructions. And the headline open models, like the largest DeepSeek and Llama variants, have hundreds of billions of parameters and need server-class hardware. "Open weights" does not mean "runs on a laptop."

The middle path: hybrid and hosted open models

You don't have to pick one side. Two options sit between local and frontier cloud:

• Hosted open-model APIs. Providers such as the ones in our Hosted LLM directory run open models for you and charge per token. You get open-model prices without buying hardware, though your data goes to the host again.
• Hybrid routing. Send easy, private or high-volume tasks to a local model, and send hard ones to a frontier API. An AI gateway or router can make that split automatic and put failover, caching and spend limits in one place.

A common pattern is a small local model for drafting and classification, with an escalation to a stronger cloud model only when the task is hard. That captures most of the savings and privacy without giving up quality where it matters.

Which one should you choose?

Choose local if privacy or offline use is a requirement, you already own capable hardware, your tasks are simple enough for a small model, or you want full control over the model version.

Choose cloud if you need the best quality, your volume is low to moderate, you have no appetite for maintenance, or you expect to switch models as better ones appear.

Four myths worth dropping

• "Local is always free." The software is. The hardware, electricity and your time are not.
• "Local is always private." Mostly, but check your tools. Some apps send telemetry, and plugins or web-search features can send data out. Verify what your setup actually does.
• "Open models are as good as GPT or Claude." The best open models are impressive and close the gap on many tasks, but the models that fit on consumer hardware are a tier below the frontier.
• "Cloud means my data trains the model." Policies differ between consumer chat apps and paid APIs. Major providers say API data isn't used for training by default, but terms and retention periods vary, so read each provider's current policy before sending sensitive data.

The verdict

For most people, start in the cloud. It costs less than the hardware debate suggests, it gives you better results, and it takes minutes to set up. Move to local, fully or partly, when you hit a concrete reason: data that can't leave, a need to work offline, hardware you already own, or a volume that makes the break-even math work. If you're unsure, a hybrid setup lets you keep both options open.

Whichever way you lean, price your actual workload before you buy anything.