A trend that accelerated through 2026 is major labs publishing the weights of their models, so anyone can run them on their own infrastructure. Alibaba announced this for its latest large model, and it is not alone.
The obvious question follows: does this mean you can stop paying for an API?
Three arguments that hold up
Data never leaves the building. This is the strongest case and often the only one you need. If you handle health records, legal documents, or anything under a confidentiality agreement, self-hosting solves a problem that a contract with a provider solves less cleanly.
No limits. No per-minute quota, no daily cap, no sudden change of terms. If your workload is uneven with sharp spikes, that matters.
Predictable cost. You pay for electricity and hardware, not per unit of use. At high volume that becomes cheaper.
Three arguments against
Quality. A model that fits on one decent GPU is not in the same class as a top commercial model. For classification and data extraction it is entirely adequate. For writing that a human is meant to read, the gap shows.
Language coverage. Smaller open models are considerably stronger in English than in languages with less training data. If your work involves anything else, expect occasional grammatical slips and text that reads like a translation. This has to be tested, not assumed.
Maintenance. Someone has to update, monitor, and fix it. If that person is not already you, an hour of their time each month costs more than the API bill.
How to test without spending anything
Install Ollama on any machine with enough memory. It runs on a plain CPU too, just slower – think several minutes for a long piece of text rather than several seconds. For overnight batch work that is perfectly acceptable.
Pull one mid-sized model, give it three real tasks from your actual work, and read the output. That test takes half an hour and answers a question no article can.
The middle ground usually wins
It does not have to be one or the other. A common setup has a local model handling the bulk simple work, with a commercial API called only where quality genuinely pays for itself. You get privacy and quality both, with a bill that stays small.