6 options · 5 criteria · checked 11 Nov 2026

The best LLM router depends on your workload

The leading LLM routers and gateways compared side by side on model coverage, latency overhead, pricing model and privacy terms, from each vendor’s own public docs. Every claim is linked and dated. No opinion rankings.

See the comparison How we compare

LLM routers and gateways, side by side

Filter by what you decide on. Each cell links to the public page it was checked against. Where a vendor publishes no figure, the cell says so and where to check.

LLM routers and gateways compared
Option	Model coverage	Latency overhead	Pricing model	Privacy terms	Fits best
You.com Inference RouterHosted routerLeading open-weight models behind one OpenAI-compatible and Anthropic-compatible endpoint, with web search and page fetching that run server-side.HostedWeb search built in	Open-weight models through /v1/chat/completions (OpenAI format) and /v1/messages (Anthropic format). GET /v1/models lists the models your key can call; access is set per account.Source, checked 11 Nov 2026	No router overhead figure is published. The docs say latency depends on the model and the length of the output; web tool rounds add search time.Source, checked 11 Nov 2026	Priced per model, no platform feeRates vary by model, with no platform fee on top. Check the rate for your model in the You.com platform before you commit.Source, checked 11 Nov 2026	Zero Data Retention is documented for the Web Search and Answer APIs on enterprise agreements. The Inference Router is not on that list yet, so ask the account team about retention for inference.Source, checked 11 Nov 2026	Agents and assistants that need open-weight models grounded in live web results Teams on the OpenAI or Anthropic SDKs who want to switch by changing a base URL and key Answers that must cite their sources, without running your own search step Weaker for: Closed frontier models outside its open-weight catalog · Contracts that need zero data retention on inference today · Self-hosting the router inside your own network
OpenRouterHosted routerA hosted marketplace router: one API and one credit balance for models from many providers, with provider routing, fallbacks and per-request privacy controls.HostedWeb search built in	Frontier models from the major labs and open-weight models, served by many providers. The full list is on the models page and the models API.Source, checked 11 Nov 2026	Built to add minimal latency, but no fixed overhead figure is published. Cold edge caches in a new region, low credit balances and failed providers can add latency.Source, checked 11 Nov 2026	Provider price plus a fee on creditsProvider prices passed through with no markup on inference, plus a fee when you buy credits (rate on openrouter.ai/pricing). Your own provider keys cost 5% of the normal price above a plan-dependent free allowance.Source, checked 11 Nov 2026	Prompts and responses are not stored unless you opt in or content is flagged as potential CSAM. Zero Data Retention can be enforced globally, per model group, per guardrail or per request. Request metadata is stored.Source, checked 11 Nov 2026	The widest hosted catalog of closed and open models on one bill Prototyping across many models without opening accounts with each lab Provider failover and price or throughput sorting without running infrastructure Weaker for: Keeping all traffic inside your own network · Spend that has to avoid a fee on credit purchases
LiteLLMSelf-hosted gatewayAn open-source Python SDK and proxy you run yourself: one OpenAI-format interface to 100+ providers, with retries, fallbacks, virtual keys and cost tracking.Self-hosted	100+ LLM providers, including OpenAI, Anthropic, Vertex AI and Bedrock, called in the OpenAI format.Source, checked 11 Nov 2026	It runs in your infrastructure, so the overhead depends on where you deploy it and what you turn on. No figure is used here; measure it in your own setup.Source, checked 11 Nov 2026	Open source: you pay providers and hostingMIT-licensed outside the repository's enterprise/ directory, which has its own licence. You pay each provider directly, plus your own hosting.Source, checked 11 Nov 2026	Requests go from your servers to the providers you configure with your own keys, so retention is set by you and by each provider's terms.Source, checked 11 Nov 2026	Teams that must keep traffic and keys inside their own infrastructure Platform teams giving many internal apps one endpoint with per-team keys and budgets Avoiding any per-token or per-credit fee from a middle layer Weaker for: Teams without capacity to run, scale and patch a proxy · Getting models you hold no provider account for
Portkey AI GatewayOpen-source gatewayAn open-source gateway you can run yourself or use hosted, with retries, fallbacks, load balancing, conditional routing and guardrails.HostedSelf-hosted	The project states routing to 1600+ language, vision, audio and image models (the vendor's figure).Source, checked 11 Nov 2026	The vendor states under 1 ms of added latency. That is the vendor's own figure, not a measurement made here.Source, checked 11 Nov 2026	Open source: you pay providers and hostingThe open-source gateway is free to run yourself; you pay providers and hosting. Hosted and enterprise plans are priced on portkey.ai.Source, checked 11 Nov 2026	Self-hosted, requests stay on your servers until they reach your providers. For the hosted gateway, check retention terms in Portkey's docs.Source, checked 11 Nov 2026	Teams who want a gateway they can self-host now and move to hosted later, or the reverse Rule-based routing and guardrails in front of provider accounts you already hold Weaker for: Buying inference through one bill without your own provider accounts · Teams that want a fully managed router with no configuration
Vercel AI GatewayHosted gatewayA hosted API for models from many providers with routing, fallbacks, prompt caching and usage monitoring, billed from prepaid credits at list price.Hosted	Models from many providers. The live list, with prices and context windows, is on the models page and a public model list API.Source, checked 11 Nov 2026	No overhead figure is published in the pages checked. Measure it against a direct provider call before you commit.Source, checked 11 Nov 2026	No markup on tokensNo markup and no platform fee on tokens, including with your own keys. You pay the provider's list price from prepaid credits; payment processing fees may apply. A free tier covers a subset of models.Source, checked 11 Nov 2026	Not verified here. Check Vercel's AI Gateway docs for its retention terms and each provider's.Source, checked 11 Nov 2026	Apps already deployed on Vercel that want one key for many providers Paying list price with no token markup, with or without your own keys Weaker for: Self-hosting the gateway · Teams that need documented retention terms before a trial
Cloudflare AI GatewayHosted gatewayA hosted gateway on Cloudflare's network for analytics, logging, caching, rate limiting, retries and model fallback in front of the providers you use.Hosted	Workers AI, OpenAI, Anthropic, Google Gemini, Amazon Bedrock, Azure OpenAI and more, through a unified OpenAI-compatible endpoint or each provider's own endpoint.Source, checked 11 Nov 2026	No overhead figure is published. Caching can serve repeated requests from Cloudflare's cache instead of the provider.Source, checked 11 Nov 2026	Provider price plus a fee on creditsCore features (analytics, caching, rate limiting) are free on all plans. Credits bought through Unified Billing carry a 5% fee; provider inference is passed through with no markup.Source, checked 11 Nov 2026	Logs are stored in Workers Logs or Legacy Logs depending on when the account made its first gateway, and can be deleted. DLP scanning is free on all plans.Source, checked 11 Nov 2026	Teams already on Cloudflare that want caching, logs and rate limits in front of providers Cutting repeat calls with a cache at the edge Weaker for: Self-hosting the gateway · Teams that want a published overhead figure before a trial

Router, gateway or model host?

They get lumped together in most lists. They solve different problems, so they’re labelled apart here.

Router

Decides which model or provider serves each request and usually bills the inference, so one key reaches many models.

Gateway

Sits in front of provider accounts you already hold and adds keys, budgets, logs, caching and fallbacks. You still pay each provider.

Model host

Runs the weights and sells the tokens. Routers send traffic to hosts; hosts aren’t ranked on this site.

Head-to-head comparisons

All comparisons →

Not sure which need decides it?

Walk through cost and fallbacks, latency, compliance, self-hosting and open-weight models, then test two candidates with a seven-point checklist.

How to choose an LLM router