LiteLLM vs Portkey vs AWS Bedrock Gateway: 2026 Guide

LiteLLM, Portkey, and AWS Bedrock Gateway all solve the same traffic problem: routing, rate limiting, and API key management for LLM deployments. Each serves a very different operational profile. LiteLLM is an open-core proxy whose own site claims 140+ providers; Portkey ships a free MIT gateway plus a managed platform with caching and observability; Bedrock Gateway is an AWS-native solution that keeps all traffic inside the AWS network. On 24 March 2026, a supply-chain attack on LiteLLM versions 1.82.7 and 1.82.8 added a security dimension to the evaluation. The right choice depends on your deployment model, DevOps maturity, and whether you’re AWS-standardized.

The conference for teams teaching AI their business

October 28 / 11 AM ET / Virtual. Join the leaders and builders defining the context layer, the missing infrastructure of the AI era.

Save Your Spot →

LiteLLM vs Portkey vs Bedrock Gateway: What’s the Difference?

All three belong to a product category called an LLM gateway: a proxy layer that sits between your applications and LLM providers, handling routing, rate limiting, API key management, observability, and caching. The category exists because calling LLM APIs directly at scale creates operational problems: key sprawl, vendor lock-in risk, no unified cost visibility, and no fallback when a provider has an outage.

The three tools diverge at deployment philosophy. LiteLLM is an open-source proxy built for teams that want maximum provider optionality and prefer to own their infrastructure. Portkey ships the same routing abstraction two ways: a free MIT gateway you host, and a managed platform you do not, for teams that want production-ready observability and caching without running the proxy themselves. AWS Bedrock Gateway is AWS’s answer for teams whose compliance and security posture requires all AI traffic to stay within the AWS network.

LiteLLM launched in 2023 as the early open-source unifier; Portkey emerged as the managed SaaS layer atop the same routing abstraction; Bedrock Gateway is AWS’s 2024-2025 answer for AWS-native enterprise teams. Two events moved the landscape in early 2026: AWS added OpenAI models (including Codex) to the Bedrock catalog in April, narrowing Bedrock Gateway’s provider gap; and on 19 February, alongside a $15M Series A led by Elevation Capital, Portkey made its basic enterprise gateway free. Portkey’s gateway itself is not a 2026 release. It has been MIT-licensed and public on GitHub since August 2023. Most existing comparisons substitute OpenRouter for Bedrock Gateway. This page covers the three tools most relevant to enterprise engineering teams choosing a production gateway in 2026.

If you’re evaluating these tools as part of a broader LLMOps strategy, the gateway decision is one layer of a larger operational picture.

What Is LiteLLM?

LiteLLM is an open-core gateway, described by its own repo as a Rust core with a Python SDK, that provides a unified OpenAI-compatible API across the 140+ providers and 1,892 models litellm.ai claims as of September 2026: Anthropic, Azure OpenAI, Vertex AI, AWS Bedrock, Cohere, Hugging Face, SageMaker, vLLM, NVIDIA NIM, and Ollama among them. The repo carries 59,263 GitHub stars as of 2026-09-21. It handles load balancing, fallback chains, spend tracking, and per-team rate limiting. The practical appeal: switch providers by changing a config value, not rewriting application code.

Licensing. LiteLLM is open-core, not plain MIT, and the distinction decides whether you can run it in production for free. The proxy is MIT-licensed. Everything under the repo’s enterprise/ directory carries the BerriAI Enterprise License, which permits production use only with a valid BerriAI subscription for the correct number of user seats. GitHub’s own licence detector returns NOASSERTION for the repo, not MIT.

Deployment and pricing. The MIT proxy is free; you pay only for the infrastructure you run it on. Enterprise adds the features regulated industries require: SSO (Okta, Azure AD, Google Workspace, any OIDC or SAML), JWT auth, RBAC at org, team and user levels, audit logs with retention policies, IP-based ACLs, key rotation, tag-based budgets and per-team logging, plus log export to your own GCS or Azure Blob storage. Enterprise Basic runs $250/month; Enterprise Premium is $30,000/year. LiteLLM also offers a self-serve 30-day enterprise trial key, emailed without a sales call. LiteLLM Enterprise lists SOC 2 Type 2 and ISO 27001; it does not claim HIPAA.

Routing and reliability. LiteLLM’s operational value is in its routing primitives: load balancing (least-busy, weighted round-robin), fallback chains, context-window-aware model escalation (auto-escalate to a larger model when a prompt exceeds context limits), retry with exponential backoff, and rate-limit-aware routing. Observability integrations include Langfuse, LangSmith, Helicone, Prometheus, and OpenTelemetry. That routing set is the reason teams reach for LiteLLM when they are managing multiple LLM providers at scale.

Security consideration: the 24 March 2026 supply-chain incident. Per LiteLLM’s own post-mortem, a file named litellm_init.pth, plus a tampered proxy_server.py, reached PyPI in versions 1.82.7 and 1.82.8 and stayed live for roughly 40 minutes from 10:39 UTC. The payload harvested environment variables, SSH keys, cloud credentials, Kubernetes tokens and database passwords, and exfiltrated them to an attacker-controlled domain. Root cause was a maintainer PyPI token exposed through a compromised dependency. LiteLLM’s remediation is to rotate credentials and pin to v1.82.6 or earlier. Teams running pinned versions or air-gapped installs were unaffected. Treat gateway patching as a recurring operational task: LiteLLM carried 27 reviewed GitHub advisories in 2026 alone, four of them rated critical.

Core capabilities

• 140+ provider integrations via unified OpenAI-compatible API (litellm.ai, Sept 2026)
• Load balancing: least-busy, weighted round-robin, cost-optimized routing
• Fallback chains and context-window-aware model escalation
• Per-team spend tracking and rate limiting; token usage dashboards
• Exact-match caching on Redis, Valkey, ElastiCache, S3, GCS or disk; semantic caching on Redis, Valkey or Qdrant
• Enterprise: SSO, JWT auth, RBAC, audit logs with retention policies, log export to GCS or Azure Blob

What Is Portkey?

Portkey routes to 1,600+ LLMs through one API, per its own gateway repo, with a built-in observability dashboard, caching, and 50+ guardrails covering PII redaction and jailbreak detection. The gateway has been MIT-licensed and public on GitHub since August 2023; what changed on 19 February 2026 is that Portkey made its basic enterprise gateway free, alongside a $15M Series A led by Elevation Capital with Lightspeed.

Deployment and pricing. Portkey publishes four tiers, and two of them are free for different reasons. Open Source is free with no log limit and you host it yourself. Developer is free forever at 10,000 recorded logs a month, with 3-day log and 30-day metric retention. Production is $49/month for 100,000 recorded logs, then $9 per additional 100k requests, with 30-day log and 90-day metric retention. Enterprise is custom-priced for 10M+ recorded logs a month, VPC hosting, dedicated support, and custom BAAs. SOC 2 Type 2, ISO 27001, GDPR and HIPAA sit on the enterprise offering, with a HIPAA report available through Portkey’s Trust Vault. The managed path means no gateway infrastructure to maintain, which matters for teams without dedicated DevOps capacity for proxy operations.

Observability and caching. Portkey’s native dashboard delivers full request logs, latency metrics, cost breakdowns, and error tracking per model, provider, team, and application. Prompt versioning and a prompt playground are built in. Caching is the feature most often mispriced in comparisons, because it splits across tiers: Portkey’s own docs make simple caching “available for all plans” and semantic caching, the vector-similarity matching that detects similar but not identical prompts, “only available on select Enterprise plans”. Portkey’s own 2023 test reported an 18% to 60% semantic cache hit rate on Q&A and RAG workloads. That is a hit rate, not a cost saving, the vendor measured itself, and it predates current models by three years. Budget the semantic cache as an Enterprise line item when you size LLM cost at the gateway layer for RAG pipelines and support agents, where many prompts are paraphrased reformulations of the same question.

Guardrails and enterprise trust signals. Portkey’s gateway repo describes 50+ AI guardrails behind one API, including PII redaction and jailbreak detection, requiring no separate vendor or custom middleware. Granular budget and rate limits apply per model, provider, team, or application. The gateway core being MIT and self-hostable since 2023 answers a recurring enterprise procurement concern: teams can run it themselves if commercial terms change.

Core capabilities

• 1,600+ LLMs through one API (gateway repo, Sept 2026), including OpenAI, Anthropic, Mistral, Gemini, Cohere, and Bedrock
• Simple caching on all plans; semantic caching (vector-similarity) on select Enterprise plans only
• Native observability dashboard: request logs, cost tracking, latency, error rates
• 50+ guardrails behind one API, including PII redaction and jailbreak detection
• Code-free routing config: fallback chains and load balancing manageable via dashboard
• SOC 2 Type 2, ISO 27001, GDPR, HIPAA, custom BAAs on the enterprise offering

What Is AWS Bedrock Gateway?

AWS Bedrock Gateway is an AWS-native LLM gateway built with Amazon API Gateway, AWS Lambda, and CloudFormation. It routes requests to the Amazon Bedrock model catalog, keeps all traffic within the AWS network boundary, and inherits AWS compliance certifications (SOC 2, HIPAA, GDPR, FedRAMP) without requiring a new vendor relationship. It is not a standalone SaaS product; it is a reference architecture pattern (aws-samples/sample-ai-gateway-for-amazon-bedrock) combined with AWS managed services.

Architecture and deployment. API Gateway handles request routing and authentication; Lambda processes and forwards requests to Bedrock; CloudFormation manages deployment. It can run as a VPC-private endpoint (zero traffic leaves the AWS network) or a regional public endpoint. There is no gateway platform fee; you pay Bedrock’s on-demand model pricing plus Lambda and API Gateway compute costs, which are minimal at low-to-mid request volumes. The architecture is positioned for organizations where all AI workloads must remain inside AWS: financial services, healthcare, defense, and regulated industries generally.

Provider catalog expansion (April 2026 update). Bedrock Gateway’s historical limitation was provider lock-in. In April 2026, AWS added OpenAI models, including Codex and managed agents, to the Bedrock catalog. The catalog now includes Anthropic Claude, Meta Llama, Mistral, Amazon Titan, Stability AI, Cohere, AI21 Labs, and OpenAI models. This narrows the provider-flexibility gap significantly for AWS-standardized teams. Developer discussions note the gateway “only implements Chat Completions,” making it less feature-complete than full-featured third-party gateways in routing and observability depth.

Observability and compliance. CloudWatch delivers usage, invocation, performance, and error-rate metrics. CloudTrail provides admin-level API event auditing. There is no dedicated LLM observability dashboard; prompt-level and token-level visibility requires additional instrumentation (typically Langfuse or OpenTelemetry). Authentication is IAM-native (SigV4 signing) with Lambda Authorizer support for JWT validation and AWS WAF integration for request filtering. The compliance posture is the key advantage: AWS Enterprise Support, full AWS SLAs, and no new vendor audit scope.

Core capabilities

• AWS-native routing to full Bedrock model catalog (including OpenAI models, added April 2026)
• IAM-native auth (SigV4) plus Lambda Authorizer for JWT; AWS WAF integration
• CloudWatch metrics and CloudTrail admin audit; inherits AWS compliance posture
• No gateway platform fee: Lambda and API Gateway compute costs only
• VPC-private deployment option: zero traffic leaves the AWS network
• Inherits SOC 2, HIPAA, GDPR, and FedRAMP certifications with no new vendor relationship

Head-to-Head Comparison

No gateway leads across every dimension. LiteLLM leads on provider breadth and self-hosted flexibility. Portkey leads on managed observability and semantic caching. Bedrock Gateway leads on AWS integration depth and compliance inheritance. The table below maps 10 operational dimensions.

A practical illustration. Consider a fintech platform team running three concurrent AI products: a support agent, a contract review tool, and an internal SQL assistant. Each touches different data classifications. LiteLLM gives them one proxy across all three, with team-level spend tracking and RBAC, but requires an internal team to manage upgrades and patching discipline (especially after the March 2026 supply-chain incident). Portkey gives them a production dashboard and guardrail hooks for PII redaction with zero gateway infrastructure to maintain, though semantic caching for the support agent’s repetitive queries would mean an Enterprise contract, not the $49 tier. If all three workloads are already in AWS, Bedrock Gateway is the path of least resistance: no new vendor, no new audit scope, same IAM policies they already govern. The right answer depends on which constraint is binding for your team.

How to Choose Between LiteLLM, Portkey, and Bedrock Gateway

The decision reduces to three constraints: how much AWS standardization you’ve committed to, how much DevOps capacity you have for gateway infrastructure, and what your observability requirements look like from day one. None of the three options are mutually exclusive; some teams combine LiteLLM and Portkey, though neither vendor documents that pairing.

Choose LiteLLM if: your team has dedicated DevOps capacity; you need to route across five or more providers; data cannot leave your network perimeter; you’re cost-sensitive at high volume (five million or more requests per month) and cannot absorb a per-call SaaS markup; or you need log export to your own GCS or Azure Blob storage. The 24 March 2026 supply-chain incident is a real operational signal: teams choosing LiteLLM should pin versions, test upgrades on a schedule, and treat gateway patching as a recurring operational task, not a set-and-forget configuration.

Choose Portkey if: your team is mid-size (20-100 engineers) without dedicated gateway DevOps; you need a production-ready dashboard on day one; you want built-in guardrails (PII redaction, jailbreak detection) without building them in-house; or you need SOC 2 Type 2 and HIPAA pre-certified, which Portkey lists and LiteLLM Enterprise does not claim for HIPAA. One caveat if caching is the reason you are looking: Portkey gates semantic caching to select Enterprise plans, so price it there, not at $49.

Choose Bedrock Gateway if: your organization is AWS-standardized; all data must stay within the AWS network boundary because of regulated-industry requirements (financial services, healthcare, defense); you already use CloudTrail, CloudWatch, and IAM as your operational baseline; you don’t need multi-cloud model routing today (or the April 2026 OpenAI-on-Bedrock addition covers your model requirements); or there is no organizational appetite for a new vendor relationship or expanded audit scope.

When to combine. Some teams run LiteLLM as the routing proxy and send traces and logs to Portkey for the observability layer. Neither vendor documents this pairing, so treat it as a pattern to test rather than a supported configuration. It gives LiteLLM’s 140+ provider breadth alongside Portkey’s dashboard. Bedrock Gateway plus LiteLLM is less common but viable for teams that route some workloads within AWS and others externally, using LiteLLM as the external-provider routing layer.

For teams managing multiple providers as part of a broader operational strategy, managing multiple LLM providers at scale covers the architectural patterns in more depth.

What LLM Gateways Don’t Govern: The Payload Problem

LiteLLM, Portkey, and AWS Bedrock Gateway all solve the traffic layer: routing requests, enforcing rate limits, logging API calls, managing keys, and caching similar prompts. None of them govern what goes through the pipe. The payload itself is opaque to all three. They can tell you that a request was made, at what cost, from which application, at what latency. They cannot tell you whether the context in that request was valid.

Five questions no LLM gateway can answer:

1. Data lineage: Where did the data in this prompt come from? Is the source table still current, or was it last refreshed three days ago?
2. Business definition consistency: Does “active customer” in this prompt mean the same thing it means in your data warehouse and your analytics layer?
3. PII completeness: Were all PII fields actually masked before this context window was assembled, not just checked at the gateway boundary?
4. Access policy enforcement at context level: Did the agent have permission to include this customer segment’s data in the prompt? The gateway received the request; it did not decide whether the context should have been built that way.
5. Cross-system lineage: Can you trace an LLM output back to the source tables and transformations that produced the input context?

Gateways sit between applications and models. The context layer for enterprise AI sits between data systems and applications, upstream of the gateway. These are different problems at different architectural layers. A perfectly configured LiteLLM deployment can still send stale, ungoverned, or policy-violating context to a model and produce plausible-sounding but incorrect outputs.

This is the gap Atlan addresses. The data graph connects data lineage, business definitions, access policies, and quality signals to the prompt assembly process, before the request reaches any LLM gateway. Teams that govern the context layer reduce both compliance risk and LLM spend more reliably than teams that optimize the gateway layer alone. For the AI context stack architecture, see Atlan’s enterprise guide. The active data governance platform is what sits above these gateways.

Real Stories from Real Customers: Governing the Layer Above the Gateway

"We're excited to build the future of AI governance with Atlan. All of the work that we did to get to a shared language at Workday can be leveraged by AI via Atlan's MCP server...as part of Atlan's AI Labs, we're co-building the semantic layer that AI needs with new constructs, like context products."

Joe DosSantos, VP of Enterprise Data & Analytics, Workday

Watch Now

"Atlan is our context operating system to cover every type of context in every system including our operational systems. For the first time we have a single source of truth for context."

Sridher Arumugham, Chief Data Analytics Officer, DigiKey

Watch Now

FAQs About LiteLLM vs Portkey vs AWS Bedrock Gateway

1. What is the main difference between LiteLLM and Portkey?

LiteLLM is an open-source self-hosted proxy optimized for provider flexibility and high-volume, DevOps-driven deployments. Portkey is a managed SaaS platform optimized for production observability, semantic caching, and minimal infrastructure overhead. LiteLLM gives you more control; Portkey gives you faster time-to-production. Both ship MIT-licensed gateway cores, but neither is free all the way up: LiteLLM’s enterprise/ directory carries a paid BerriAI licence, and Portkey’s managed platform features (observability dashboard, semantic caching, and guardrails) require a subscription.

2. What is AWS Bedrock Gateway and how does it work?

AWS Bedrock Gateway is a reference architecture built with Amazon API Gateway, AWS Lambda, and CloudFormation that routes LLM requests to the Amazon Bedrock model catalog. It keeps all traffic within the AWS network, uses IAM (SigV4) for authentication, and inherits AWS compliance certifications without requiring a new vendor relationship. It does not offer semantic caching or a dedicated LLM observability dashboard. In April 2026, AWS added OpenAI models to the Bedrock catalog, narrowing the provider-flexibility limitation for AWS-standardized teams.

3. Is LiteLLM free to use in production?

The LiteLLM proxy is MIT-licensed and free to run; you pay only for the infrastructure you run it on. Everything under the repo’s enterprise/ directory is different: it carries the BerriAI Enterprise License, which permits production use only with a paid subscription for the correct seat count. Enterprise features (SSO, RBAC, audit logs, SLA support) run $250/month (Basic) or $30,000/year (Premium), and LiteLLM emails a self-serve 30-day enterprise trial key without a sales call. The 24 March 2026 supply-chain attack affected only PyPI packages at versions 1.82.7 and 1.82.8.

4. Does Portkey support self-hosted deployment?

Yes. Portkey’s gateway has been MIT-licensed and public on GitHub since August 2023, and in February 2026 Portkey made its basic enterprise gateway free. The managed platform is a subscription: the observability dashboard and 30-day log retention come with the $49 Production tier, while semantic caching is available only on select Enterprise plans.

5. Which LLM gateway is best for enterprise compliance: SOC 2, HIPAA, FedRAMP?

For FedRAMP, Bedrock Gateway is the only option of the three (inheriting AWS’s FedRAMP certification). Portkey lists SOC 2 Type 2, ISO 27001, GDPR and HIPAA, with custom BAAs and a HIPAA report through its Trust Vault. LiteLLM Enterprise lists SOC 2 Type 2 and ISO 27001, and does not claim HIPAA anywhere. Bedrock Gateway inherits all AWS compliance certifications with no new vendor audit scope.

6. What is semantic caching in LLM gateways?

Semantic caching uses vector-similarity matching to detect prompts that are similar but not identical to cached prompts, and returns a cached response instead of calling the model. Unlike exact-match caching, it handles paraphrased or lightly reformulated queries. Portkey’s docs put simple caching on all plans and semantic caching on select Enterprise plans only. LiteLLM documents semantic caching in the open-source proxy, on Redis, Valkey or Qdrant backends, alongside exact-match caching on Redis, S3, GCS and disk.

7. Can you use LiteLLM and Portkey together?

Yes. Running LiteLLM as the routing proxy with Portkey handling observability is a pattern some teams run, but neither vendor documents it, so test it rather than assuming support. LiteLLM handles provider routing and load balancing; Portkey’s dashboard receives the traces and logs. This gives teams LiteLLM’s 140+ provider coverage alongside Portkey’s observability dashboard without maintaining a separate observability stack.

8. What are the limitations of LLM gateways for enterprise governance?

LLM gateways handle the traffic layer: routing, rate limiting, key management, and API-level logging. They do not govern what goes into the prompt payload. Data lineage, business definition consistency, PII masking completeness, access policy enforcement at context level, and cross-system output traceability are all context-layer problems that sit upstream of the gateway, between data systems and the applications that assemble prompts.

Sources

1. LiteLLM official documentation
2. LiteLLM Enterprise features
3. Portkey pricing page
4. AWS Bedrock AgentCore Gateway documentation
5. AWS Architecture Blog: Building an AI gateway to Amazon Bedrock
6. Portkey: simple and semantic caching
7. Portkey Series A announcement, February 2026
8. AWS What’s New: OpenAI models on Bedrock (April 2026)
9. LiteLLM security update: March 2026 supply-chain incident
10. BerriAI Enterprise License
11. Atlan: AI Gateway and LLM Gateway