AI Gateways

An AI gateway is a service that sits between your application and one or more LLM providers and exposes a single API endpoint. Instead of integrating OpenAI, Anthropic, Google and others separately, your code talks to the gateway, which forwards each request to the model you choose. Beyond routing, gateways typically add automatic retries and failover when a provider is down, response caching, rate limiting, budget controls, usage logging and observability, and central management of API keys. Some are hosted services, such as OpenRouter, Vercel AI Gateway and Cloudflare AI Gateway, while others, such as LiteLLM, are open source proxies you run yourself. The cards on this page compare them by type, deployment and pricing.

For a prototype that uses one provider, calling the API directly is simpler and perfectly fine. A gateway starts to pay off when you use several models or providers, when reliability matters because you want automatic fallback if one provider has an outage, or when you need cost visibility across teams. It also helps with governance: one place to set budgets, rotate keys and log requests. The trade-offs are an extra network hop, a possible fee or markup, and one more component to depend on. If you only need to switch between a couple of models occasionally, a lightweight library may be enough, so let your real requirements decide rather than adding the layer in advance.

The terms overlap, and many products do both. A router focuses on access and model choice: one account and one bill that reaches hundreds of models from many providers, sometimes choosing the cheapest or fastest host for you. A gateway focuses on control: caching, guardrails, rate limits, budgets, logging and observability for traffic you already send, often with your own provider keys. OpenRouter is the best-known aggregator, Portkey and Helicone lean toward observability and control, and Cloudflare and Vercel provide gateways tied to their platforms. Choose by your main need, which is either broad model access or tighter operational control. The groups on this page separate them to make that decision easier.

Both are possible, and the size differs by product. A hosted gateway adds a network hop, typically measured in tens of milliseconds, which is small next to the seconds a model takes to generate an answer, and caching can make repeated requests faster and cheaper. On price, some gateways pass provider rates through and charge a platform fee or a markup on credits, while self-hosted open source options such as LiteLLM have no per-request fee but cost your own infrastructure and maintenance time. Savings can also appear, because caching, routing to cheaper models and budget limits cut spending. Check each gateway's pricing line on this page, and measure latency yourself with your real traffic.

Match the gateway to your situation. If you want one key and many models with little setup, an aggregator like OpenRouter is the quickest. If your app already runs on Vercel or Cloudflare, their built-in gateways integrate with the platform and its billing. If you need full control, data residency or no vendor fees, a self-hosted open source proxy such as LiteLLM or Helicone fits, at the cost of running it. Large organizations that already use an API management layer often choose an enterprise gateway like Kong. Compare the options in the groups above, then try one on a small project, since most offer a free tier and use an OpenAI-compatible API that is easy to swap.