AI operations
Compare LLM gateways for unified model access, routing, fallbacks, budgets, observability, provider keys, self-hosting, and production AI operations.
Comparison
| Option | Best for | Strengths | Tradeoffs | Use when |
|---|---|---|---|---|
| LiteLLM | Self-hosted or team-controlled OpenAI-compatible gateway workflows | Open-source gateway/proxy posture with many provider integrations. | Your team owns deployment, upgrades, and operational reliability when self-hosted. | You want unified provider access with more control over infrastructure. |
| Portkey | Enterprise gateway, observability, retries, fallbacks, governance, and security controls | Gateway plus production controls and visibility for GenAI apps. | Platform fit depends on compliance, pricing, and whether its control plane matches your architecture. | You want a managed production AI gateway with enterprise operations features. |
| OpenRouter | Accessing many models through a unified API with automatic fallbacks and provider breadth | Fast path to model variety and OpenAI-compatible integration patterns. | Less ideal when you need full internal control over provider contracts or data paths. | Model exploration and multi-provider access are more important than self-hosting. |
| Vercel AI Gateway | Vercel and web app teams that want unified model access, budgets, monitoring, and fallbacks | Tight fit with Vercel AI SDK and web application deployment. | Best fit is strongest when your application stack already lives near Vercel. | You build AI web apps and want gateway behavior inside the Vercel ecosystem. |
Most teams start by calling one model directly. Gateways appear when the product needs provider switching, fallbacks, cost controls, centralized keys, rate limits, audit logs, or model experiments across many applications.
A gateway is production infrastructure. Compare uptime, latency overhead, supported providers, retry behavior, budget controls, observability, data handling, key management, and export paths.
Some gateways include observability; some observability platforms can proxy requests. Decide whether you need routing first, tracing first, or a combined control plane.
Decision Rules
Use LiteLLM when self-hosted control and provider compatibility matter.
Use Portkey when enterprise gateway controls and observability are central.
Use OpenRouter when broad model access and fast experimentation matter.
Use Vercel AI Gateway when your AI app stack is already Vercel-centered.
Related Guides
Compare gateway needs with tracing and evaluation needs.
OpenUse budgets and routing to control AI API spend.
OpenCompare tracing and eval platforms alongside gateways.
OpenBudget and route requests based on real economics.
OpenConnect gateway decisions to agent orchestration.
OpenChinese Archive
Topic Hubs
Industry Pages
FAQ
An LLM gateway is a routing layer between your app and model providers. It can centralize keys, budgets, retries, fallbacks, logging, and provider switching.
Use one when multiple apps or providers need shared controls. For a small prototype calling one model, direct API access can be simpler.
No. A gateway routes and controls requests. Observability explains what happened. Some platforms combine both, but the jobs are different.
Source Links
Reference
Official LiteLLM proxy and gateway documentation.
OpenReference
Official Portkey AI Gateway documentation.
OpenReference
Official OpenRouter unified API quickstart.
OpenReference
Official Vercel AI Gateway documentation.
OpenThe strongest decision is always local to your workflow. Save the vendor links, define a representative task, record the exact prompt or command, and compare the final evidence instead of the marketing claim.
Return to the AI learning map