Hosted LLM

A hosted open-weight LLM is an open model, such as Llama, Qwen, DeepSeek or Mistral, that a third-party company runs on its own GPUs and sells to you per token through an API. You get the flexibility of open models without buying hardware or managing servers. Providers such as Groq, Together AI, Fireworks AI, DeepInfra, Cerebras and SambaNova compete on price, speed and reliability, so the same model can cost very different amounts depending on the host. Most expose an OpenAI-compatible API, which means switching often needs only a changed base URL and API key. This page lists the main hosts so you can compare them and pick one that fits your workload.

The billing model is the same, because you pay per million input tokens and per million output tokens, but the rates are often much lower. Since an open model can be served by many companies, they compete, and prices for a given model are frequently a fraction of what the closed frontier models cost. Some hosts also offer a free tier, batch discounts or cheaper cached input. Prices differ between hosts for the identical model, so it pays to compare. Quality is not identical either: open models trail the very best proprietary ones on the hardest tasks. Use the price comparison to see how a hosted model stacks up against closed models for your own typical prompt.

There is no single winner, because it depends on the model and the metric. Companies with custom inference chips, such as Groq, Cerebras and SambaNova, are known for very high tokens-per-second speeds, which suits real-time and agent workloads. General GPU clouds such as Together AI, Fireworks AI, DeepInfra and Novita AI often compete hardest on price per million tokens and offer large model catalogs. Aggregators such as OpenRouter route requests across several hosts for you. Speed, price, context length, quantization level and rate limits all vary by model, and they change often. Compare the cards on this page, then test your own prompt on two providers before you commit.

Safety depends on the provider's terms, so read them before you send anything sensitive. Check whether prompts are stored, for how long, and whether they are used for training. Reputable hosts state a no-training policy for API traffic, offer data processing agreements and publish security certifications, and some offer zero data retention or region choices. Because these companies are smaller and more varied than the big labs, the differences are larger, so verify instead of assuming. If your data is highly confidential, the stricter alternative is running the same open model yourself; see Run Your LLM. For ordinary public or low-risk content, a hosted API is typically a reasonable trade-off.

Usually yes. Most hosted providers implement an OpenAI-compatible chat completions API, so in many cases you only change the base URL, the API key and the model name. Model names differ between hosts, and features such as tool calling, JSON mode, vision and prompt caching are not supported equally everywhere, so test the features you rely on. To avoid lock-in entirely, put an AI gateway or router in front, which gives you a single endpoint and lets you move traffic or add automatic fallback if a host goes down. The AI Gateways page compares those options, including OpenRouter, LiteLLM and Portkey.