← Back to daily report

llm-gateway.md

Changed on 2026-06-25 04:28:50 EST

Unified Diff

--- a/llm-gateway.md
+++ b/llm-gateway.md
@@ -2,212 +2,89 @@
 > Fetch the complete documentation index at: https://code.claude.com/docs/llms.txt
 > Use this file to discover all available pages before exploring further.
 
-# LLM gateway configuration
+# LLM gateways
 
-> Learn how to configure Claude Code to work with LLM gateway solutions. Covers gateway requirements, authentication configuration, model selection, and provider-specific endpoint setup.
+> Route Claude Code through an LLM gateway for centralized authentication, usage tracking, and cost controls. Covers connecting Claude Code to a gateway, rolling one out for your organization, what Claude Code sends to a gateway, and how gateways interact with claude.ai subscriptions.
 
-LLM gateways provide a centralized proxy layer between Claude Code and model providers, often providing:
+An LLM gateway is a proxy your organization runs between Claude Code and a model provider. Claude Code sends API traffic to the gateway, and the gateway forwards it to the provider using a credential your organization controls.
 
-* **Centralized authentication** - Single point for API key management
-* **Usage tracking** - Monitor usage across teams and projects
-* **Cost controls** - Implement budgets and rate limits
-* **Audit logging** - Track all model interactions for compliance
-* **Model routing** - Switch between providers without code changes
+This page covers:
 
-This page covers gateway requirements and configuration for the Claude Code CLI. Enterprise Desktop deployments can configure gateway providers via [managed settings](https://support.claude.com/en/articles/12622667-enterprise-configuration). The Claude Desktop app can also run against a self-hosted gateway through the [Cowork on 3P research preview](https://claude.com/docs/cowork/3p/gateway), which uses its own configuration keys.
-
-## Gateway requirements
-
-For an LLM gateway to work with Claude Code, it must meet the following requirements:
-
-**API format**
-
-The gateway must expose to clients at least one of the following API formats:
-
-1. **Anthropic Messages**: `/v1/messages`, `/v1/messages/count_tokens`
-   * Must forward request headers: `anthropic-beta`, `anthropic-version`
-
-2. **Bedrock InvokeModel**: `/invoke`, `/invoke-with-response-stream`
-   * Must preserve request body fields: `anthropic_beta`, `anthropic_version`
-
-3. **Vertex rawPredict**: `:rawPredict`, `:streamRawPredict`, `/count-tokens:rawPredict`
-   * Must forward request headers: `anthropic-beta`, `anthropic-version`
-
-Failure to forward headers or preserve body fields may result in reduced functionality or inability to use Claude Code features.
+* [What a gateway provides](#what-a-gateway-provides)
+* [How routing and credentials work](#how-a-gateway-works)
+* [The steps to roll one out](#roll-out-a-gateway)
+* [How gateways interact with claude.ai subscriptions](#subscriptions-and-gateways)
+* [What's configured separately from the gateway](#configure-separately-from-the-gateway)
 
 <Note>
-  Claude Code determines which features to enable based on the API format. When using the Anthropic Messages format with Bedrock or Vertex, you may need to set environment variable `CLAUDE_CODE_DISABLE_EXPERIMENTAL_BETAS=1`.
+  - If you're a developer connecting to an existing gateway: [connect Claude Code to your gateway](/en/llm-gateway-connect)
+  - If you're an admin rolling out a gateway for your organization: [deploy and distribute a gateway](/en/llm-gateway-rollout)
+  - If you're configuring a gateway product: the [gateway protocol reference](/en/llm-gateway-protocol)
 </Note>
 
-**Request headers**
+## What a gateway provides
 
-Claude Code includes the following headers on API requests:
+A gateway gives your organization one place to manage:
 
-| Header                          | Description                                                                                                                                                                                                                                                              |
-| :------------------------------ | :----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| `X-Claude-Code-Session-Id`      | A unique identifier for the current Claude Code session. Proxies can use this to aggregate all API requests from a single session without parsing the request body.                                                                                                      |
-| `X-Claude-Code-Agent-Id`        | Identifier of the subagent or teammate that issued the request. Your proxy can use this to attribute API cost to individual parallel subagents within a session, without parsing the request body. Present only for requests made by an in-process subagent or teammate. |
-| `X-Claude-Code-Parent-Agent-Id` | Identifier of the agent that spawned the agent making the request. Use this with `X-Claude-Code-Agent-Id` to attribute API costs across nested agents in your proxy. Present only when the requesting agent was itself spawned by another agent.                         |
+* **Credentials**: the provider key stays server-side; developers hold gateway credentials instead
+* **Usage tracking**: attribute usage by developer or team, regardless of which provider serves the request
+* **Cost controls**: enforce budgets and rate limits in one place
+* **Audit logging**: log every model request for compliance
+* **Provider switching**: change the provider in gateway configuration, without touching developer machines
 
-Both agent ID headers are ephemeral per-spawn identifiers, not persistent user or device IDs.
+All of these except provider switching apply whether the upstream is Anthropic's API or a [cloud provider](/en/third-party-integrations).
 
-Claude Code also prepends a short attribution block to the system prompt containing the client version and a fingerprint derived from the conversation. The Anthropic API strips this block before processing, so it does not affect first-party prompt caching. If your gateway implements its own prompt cache keyed on the full request body, set [`CLAUDE_CODE_ATTRIBUTION_HEADER=0`](/en/env-vars) to omit it.
+The tradeoff is that the gateway becomes infrastructure your organization operates. Claude Code adds capabilities with each release, and a gateway that doesn't forward them breaks the corresponding features, so the gateway product needs to be kept updated as Claude Code evolves. The [gateway protocol reference](/en/llm-gateway-protocol) covers what to forward.
 
-## Configuration
+## How a gateway works
 
-### Model selection
+By default, Claude Code sends requests directly to Anthropic's API at `api.anthropic.com`. To route through a gateway, set `ANTHROPIC_BASE_URL` to the gateway's address; Claude Code sends the same requests there instead. The gateway authenticates the developer, attaches your organization's provider credential, and forwards each request to whichever provider it's configured for.
 
-By default, Claude Code uses standard model names for the selected API format.
+`ANTHROPIC_BASE_URL` is the address variable for most gateways. A gateway that fronts a specific cloud provider, such as Bedrock, Vertex, Foundry, or the Claude Platform on AWS, uses that provider's base URL variable instead; [API formats](/en/llm-gateway-protocol#api-formats) lists which variable goes with each configuration.
 
-When `ANTHROPIC_BASE_URL` points at a gateway that exposes the Anthropic Messages format, Claude Code can query the gateway's `/v1/models` endpoint at startup and add the returned models to the `/model` picker. Set `CLAUDE_CODE_ENABLE_GATEWAY_MODEL_DISCOVERY=1` to enable this. Discovery is off by default so that gateways backed by a shared API key do not surface every model the key can access to every user. Each discovered entry is labeled "From gateway" and uses the `display_name` field from the response when one is provided. This requires Claude Code v2.1.129 or later.
+<Frame>
+  <img src="https://mintcdn.com/claude-code/zIcIE_SQv4Z0Zbhc/images/llm-gateway-flow.svg?fit=max&auto=format&n=zIcIE_SQv4Z0Zbhc&q=85&s=490607d033d235694efb49a73a5b9e4b" alt="Diagram showing Claude Code routing through an LLM gateway. In a developer machines zone, the Claude Code CLI, VS Code extension, and CI or Agent SDK clients send requests to the gateway, with the base URL variable for the gateway's API format pointing at it and each developer holding a per-developer credential, and the desktop app reaches the same gateway through organization-distributed configuration. In a zone labeled your infrastructure, the LLM gateway handles authentication, usage tracking, budgets, and routing, and forwards requests with your organization's credential. In a model providers zone, a solid arrow leads to the provider you configure, shown as the Anthropic API, and dashed arrows lead to other provider options, illustrated with Amazon Bedrock, Google Vertex AI, and Microsoft Foundry as examples." width="780" height="322" data-path="images/llm-gateway-flow.svg" />
+</Frame>
 
-Discovery applies only to the Anthropic Messages format. It does not run for Bedrock or Vertex pass-through endpoints, and it does not run when `ANTHROPIC_BASE_URL` is unset or points at `api.anthropic.com`.
+Two kinds of credential are involved:
 
-The discovery request authenticates the same way as inference requests: it sends `ANTHROPIC_AUTH_TOKEN` as a bearer token, or `ANTHROPIC_API_KEY` as the `x-api-key` header when no auth token is set, along with any headers from `ANTHROPIC_CUSTOM_HEADERS`. Only models whose ID begins with `claude` or `anthropic` are added to the picker. Results are cached to `~/.claude/cache/gateway-models.json` and refreshed on each startup. If the request fails or the gateway does not implement `/v1/models`, the picker falls back to the cached list from the previous startup or to the built-in model list.
+* **Developer credentials**: each developer holds their own, issued by the gateway. It authenticates them to the gateway and identifies them in usage tracking
+* **Provider credential**: the gateway holds one credential for your provider account, shared by all forwarded traffic. You don't provision provider keys per developer
 
-If your gateway uses model names that do not match the discovery filter, use the environment variables documented in [Model configuration](/en/model-config) to add them manually.
+The gateway forwards each request to the provider you configure, such as the Anthropic API, [Amazon Bedrock](/en/amazon-bedrock), [Google Vertex AI](/en/google-vertex-ai), [Microsoft Foundry](/en/microsoft-foundry), or the [Claude Platform on AWS](/en/claude-platform-on-aws). Because Claude Code talks only to the gateway, the provider choice is the gateway's configuration, not the client's.
 
-## LiteLLM configuration
+## Roll out a gateway
 
-<Warning>
-  LiteLLM PyPI versions 1.82.7 and 1.82.8 were compromised with credential-stealing malware. Do not install these versions. If you have already installed them:
+When you're ready to roll out an LLM gateway to your organization, the sequence is the same whichever gateway product you choose:
 
-  * Remove the package
-  * Rotate all credentials on affected systems
-  * Follow the remediation steps in [BerriAI/litellm#24518](https://github.com/BerriAI/litellm/issues/24518)
+1. Deploy the gateway and give it your provider credential, so it can authenticate the requests it forwards.
+2. Issue each developer a gateway credential, so usage is attributed to the developer and offboarding revokes one credential.
+3. Distribute the configuration through a [managed settings file](/en/settings#settings-files) and your secrets tooling, so every machine receives the base URL and a credential. When both are distributed, developers don't configure anything. If you don't have settings distribution in place, developers follow the [connect page](/en/llm-gateway-connect) to set the variables themselves.
+4. Have each developer [check for the configuration in Claude Code](/en/llm-gateway-connect#check-for-an-existing-configuration), so distribution problems surface before they depend on the gateway.
 
-  LiteLLM is a third-party proxy service. Anthropic doesn't endorse, maintain, or audit LiteLLM's security or functionality. This guide is provided for informational purposes and may become outdated. Use at your own discretion.
-</Warning>
+[Roll out an LLM gateway for your organization](/en/llm-gateway-rollout) walks each step and shows the configuration files to distribute at each one. The gateway is one part of organization setup; for policy enforcement, usage visibility, and data handling decisions, see [Set up Claude Code for your organization](/en/admin-setup).
 
-### Prerequisites
+## Third-party gateways
 
-* Claude Code updated to the latest version
-* LiteLLM Proxy Server deployed and accessible
-* Access to Claude models through your chosen provider
+Any gateway that exposes a [supported API format](/en/llm-gateway-protocol#api-formats) works. Anthropic doesn't endorse, maintain, or audit third-party gateway products. Deploy them following their own documentation, then complete the Claude Code side of the rollout with the [rollout steps](/en/llm-gateway-rollout).
 
-### Basic LiteLLM setup
+## Subscriptions and gateways
 
-**Configure Claude Code**:
+While a [gateway credential variable](/en/llm-gateway-connect#set-the-credential-variable) or `apiKeyHelper` is active, a developer's claude.ai subscription isn't used: the credential replaces the subscription login for that session, and the subscription's usage limits don't apply. That traffic is billed per token to whoever owns the credential the gateway forwards, such as your organization's Anthropic Console account, or your Bedrock, Vertex, or Foundry account when the gateway routes there.
 
-#### Authentication methods
+Setting only `ANTHROPIC_BASE_URL`, without a gateway credential, doesn't replace the subscription. Requests still route through the gateway, but a saved claude.ai login remains the active credential, so its usage limits and billing apply. Gateways that pass this traffic on to Anthropic must forward the OAuth capability in `anthropic-beta`; see the [request headers reference](/en/llm-gateway-protocol#request-headers).
 
-##### Static API key
+## Configure separately from the gateway
 
-Simplest method using a fixed API key:
+A gateway determines where model API requests are sent. Model selection, the rest of Claude Code's network traffic, and corporate proxies are configured separately:
 
-```bash theme={null}
-# Set in environment
-export ANTHROPIC_AUTH_TOKEN=sk-litellm-static-key
+* **Model selection**: the base URL decides where requests go, not which model answers them. Pick the model with the `/model` command or the model environment variables; see [how to set your model](/en/model-config#setting-your-model)
+* **Client-side traffic**: version checks and optional client telemetry, both disabled with [`CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC`](/en/env-vars), and login traffic when a claude.ai or Console login is in use, go to Anthropic's update and authentication endpoints rather than the gateway. See [network access requirements](/en/network-config#network-access-requirements) for the domains
+* **Corporate proxies**: a proxy set with `HTTPS_PROXY` sits between Claude Code and every server it talks to, including the gateway. If your network requires a proxy, configure both; see [proxy configuration](/en/network-config#proxy-configuration)
 
-# Or in Claude Code settings
-{
-  "env": {
-    "ANTHROPIC_AUTH_TOKEN": "sk-litellm-static-key"
-  }
-}
-```
+## Related pages
 
-This value will be sent as the `Authorization` header.
-
-##### Dynamic API key with helper
-
-For rotating keys or per-user authentication:
-
-1. Create an API key helper script:
-
-```bash theme={null}
-#!/bin/bash
-# ~/bin/get-litellm-key.sh
-
-# Example: Fetch key from vault
-vault kv get -field=api_key secret/litellm/claude-code
-
-# Example: Generate JWT token
-jwt encode \
-  --secret="${JWT_SECRET}" \
-  --exp="+1h" \
-  '{"user":"'${USER}'","team":"engineering"}'
-```
-
-2. Configure Claude Code settings to use the helper:
-
-```json theme={null}
-{
-  "apiKeyHelper": "~/bin/get-litellm-key.sh"
-}
-```
-
-3. Set token refresh interval:
-
-```bash theme={null}
-# Refresh every hour (3600000 ms)
-export CLAUDE_CODE_API_KEY_HELPER_TTL_MS=3600000
-```
-
-This value will be sent as `Authorization` and `X-Api-Key` headers. The `apiKeyHelper` has lower precedence than `ANTHROPIC_AUTH_TOKEN` or `ANTHROPIC_API_KEY`.
-
-#### Unified endpoint (recommended)
-
-Using LiteLLM's [Anthropic format endpoint](https://docs.litellm.ai/docs/anthropic_unified):
-
-```bash theme={null}
-export ANTHROPIC_BASE_URL=https://litellm-server:4000
-```
-
-**Benefits of the unified endpoint over pass-through endpoints:**
-
-* Load balancing
-* Fallbacks
-* Consistent support for cost tracking and end-user tracking
-
-#### Provider-specific pass-through endpoints (alternative)
-
-##### Claude API through LiteLLM
-
-Using [pass-through endpoint](https://docs.litellm.ai/docs/pass_through/anthropic_completion):
-
-```bash theme={null}
-export ANTHROPIC_BASE_URL=https://litellm-server:4000/anthropic
-```
-
-##### Amazon Bedrock through LiteLLM
-
-Using [pass-through endpoint](https://docs.litellm.ai/docs/pass_through/bedrock):
-
-```bash theme={null}
-export ANTHROPIC_BEDROCK_BASE_URL=https://litellm-server:4000/bedrock
-export CLAUDE_CODE_SKIP_BEDROCK_AUTH=1
-export CLAUDE_CODE_USE_BEDROCK=1
-```
-
-##### Google Vertex AI through LiteLLM
-
-Using [pass-through endpoint](https://docs.litellm.ai/docs/pass_through/vertex_ai):
-
-```bash theme={null}
-export ANTHROPIC_VERTEX_BASE_URL=https://litellm-server:4000/vertex_ai/v1
-export ANTHROPIC_VERTEX_PROJECT_ID=your-gcp-project-id
-export CLAUDE_CODE_SKIP_VERTEX_AUTH=1
-export CLAUDE_CODE_USE_VERTEX=1
-export CLOUD_ML_REGION=us-east5
-```
-
-##### Claude Platform on AWS through a gateway
-
-Route to a gateway that forwards to the [Claude Platform on AWS](/en/claude-platform-on-aws) endpoint:
-
-```bash theme={null}
-export ANTHROPIC_AWS_BASE_URL=https://litellm-server:4000/anthropic-aws
-export ANTHROPIC_AWS_WORKSPACE_ID=wrkspc_01ABCDEFGHIJKLMN
-export CLAUDE_CODE_SKIP_ANTHROPIC_AWS_AUTH=1
-export CLAUDE_CODE_USE_ANTHROPIC_AWS=1
-```
-
-For more detailed information, refer to the [LiteLLM documentation](https://docs.litellm.ai/).
-
-## Additional resources
-
-* [LiteLLM documentation](https://docs.litellm.ai/)
-* [Claude Code settings](/en/settings)
-* [Enterprise network configuration](/en/network-config)
-* [Third-party integrations overview](/en/third-party-integrations)
+* [Connect Claude Code to an LLM gateway](/en/llm-gateway-connect): set the base URL and credential on your own machine, with per-surface configuration and a troubleshooting table
+* [Roll out an LLM gateway for your organization](/en/llm-gateway-rollout): the admin checklist for deploying a gateway, issuing developer credentials, and distributing managed settings
+* [Gateway protocol reference](/en/llm-gateway-protocol): what Claude Code sends to a gateway, for operators configuring one, covering endpoints, headers to forward, and feature pass-through
+* [Set up Claude Code for your organization](/en/admin-setup): the wider rollout decisions a gateway is one part of, including policy enforcement and usage visibility