Model routers and AI gateways
A working catalog of the services and open-source projects used to route, govern, and observe model traffic.
LiteLLM
LiteLLM is an open-source proxy and gateway that normalizes model APIs behind an OpenAI-compatible interface. It combines broad provider support with routing, virtual keys, budgets, caching, observability, and guardrail integrations.
- Deployment
- Self-hosted and managed
- Model access
- Broad multi-provider catalog
Portkey
Portkey is a production AI gateway that combines model access, routing, observability, governance, prompt management, and guardrails. It is designed to give platform teams a shared control plane for multiple AI applications.
- Deployment
- Managed and enterprise deployment
- Model access
- Broad multi-provider catalog
Databricks Unity AI Gateway
Databricks Unity AI Gateway centralizes access, governance, monitoring, and policy enforcement for AI traffic within the Databricks platform. Its strongest fit is with teams already using Unity Catalog and Mosaic AI infrastructure.
- Deployment
- Managed within Databricks
- Model access
- Databricks serving endpoints and supported providers
OpenRouter
OpenRouter provides one API and billing layer across a large catalog of models and inference providers. It can route requests between providers according to availability, price, throughput, latency, data policies, and application preferences.
- Deployment
- Managed service
- Model access
- Hundreds of hosted models and providers
Requesty
Requesty is a managed AI gateway for routing, monitoring, cost control, and policy enforcement across model providers. It supports production features such as regional failover, semantic caching, sensitive-data controls, and guardrails.
- Deployment
- Managed service
- Model access
- Broad multi-provider catalog
Vercel AI Gateway
Vercel AI Gateway offers a consistent endpoint for accessing models from multiple providers. It is tightly integrated with the Vercel AI SDK and emphasizes low-friction model access, provider fallback, usage visibility, and managed operations.
- Deployment
- Managed service
- Model access
- Broad multimodal model catalog
Kong AI Gateway
Kong AI Gateway extends Kong's API management platform to LLM, MCP, and agent traffic. It brings model traffic into an established gateway environment with authentication, routing, rate controls, analytics, and AI-specific policy plugins.
- Deployment
- Managed and self-managed
- Model access
- Multiple providers through Kong plugins
Cloudflare AI Gateway
Cloudflare AI Gateway sits in front of AI providers to add analytics, caching, rate controls, fallback, and policy enforcement. It is delivered through Cloudflare's global network and integrates naturally with Workers AI and existing Cloudflare infrastructure.
- Deployment
- Managed edge service
- Model access
- Multiple hosted providers and Workers AI
Merge Gateway
Merge Gateway is an embedded AI gateway for products that need to give their customers model choice, routing, spend controls, and governance. It includes deterministic and intelligent routing alongside DLP and prompt-injection protection.
- Deployment
- Managed and embedded
- Model access
- Multiple model providers
Not Diamond
Not Diamond focuses on learned model selection. Its router predicts which model is likely to perform best for a request, with particular emphasis on quality, cost, latency, and coding-agent workloads.
- Deployment
- Managed service
- Model access
- Supported frontier and open models
TrustedRouter
TrustedRouter is a privacy-focused AI gateway built around no-content-log infrastructure, zero-data-retention routes, regional routing, and verifiable deployment controls. It also provides unified access to a broad model catalog.
- Deployment
- Managed and self-hosted
- Model access
- Broad multi-provider catalog
Envoy AI Gateway
Envoy AI Gateway is a Kubernetes-oriented, open-source gateway for AI traffic. It builds on Envoy and Gateway API concepts to provide provider integration, routing, traffic management, MCP support, and infrastructure-level observability.
- Deployment
- Self-hosted on Kubernetes
- Model access
- Multiple model providers
Odock
Odock is an AI and MCP governance gateway with model access, policy routing, virtual keys, budgets, quotas, guardrails, and audit controls. It treats model and MCP traffic as part of one governed runtime layer.
- Deployment
- Managed gateway
- Model access
- Multiple model providers and MCP servers