LLM Gateways

Connect Fireworks to LiteLLM, Portkey, Vercel AI Gateway, Helicone, Keywords AI, AISIX, Requesty, OpenRouter, or Cloudflare AI Gateway

Gateways integrate with Fireworks in two ways. Some connect to your Fireworks account and preserve Fireworks model IDs. Others expose selected Fireworks models through a gateway-managed catalog.

​

Check whether your gateway can use Nexus Auto

To call auto, your gateway must:

1. Connect to your Fireworks account.
2. Send the auto model ID unchanged.
3. Send your Fireworks API key.
4. Provide an Anthropic or OpenAI credential if you want one of those models.

A gateway-managed catalog can use Nexus Auto only when that catalog lists auto. Fireworks serving some of the catalog’s models does not make auto available.

​

Connect your Fireworks account

These gateways connect to your Fireworks account and keep Fireworks model IDs: LiteLLM, Portkey, Keywords AI, AISIX / API7, Helicone, and Cloudflare AI Gateway.

​

Use a gateway-managed model catalog

Vercel AI Gateway, OpenRouter, Requesty, and Helicone AI Gateway expose their own model catalogs. Fireworks may serve a request, but you can call only the models the gateway lists.

​

LiteLLM Proxy

Use LiteLLM as a shared gateway for Fireworks serverless models, Nexus Auto, and deployments. Developers call one OpenAI-compatible chat completions API, and LiteLLM manages the Fireworks connection and client access.

1. Install LiteLLM Proxy

   You need a Nexus API key (fw_...).

   pip install "litellm[proxy]"
   
2. Configure a Fireworks model

   Add each model to config.yaml. Keep model_name short, and use fireworks_ai/ plus the full Fireworks path in litellm_params.model:

   config.yaml

   model_list:
     - model_name: glm-latest
       litellm_params:
         model: fireworks_ai/accounts/fireworks/routers/glm-latest
         api_key: os.environ/FIREWORKS_API_KEY
         api_base: https://api.fireworks.ai/inference/v1
   

   A short router ID such as fireworks_ai/auto returns 404 Model not found. Use the full path.

3. Start the proxy

   litellm --config config.yaml
   
4. Call a model

   If LiteLLM virtual keys are configured, clients authenticate with a virtual key:

   curl http://localhost:4000/chat/completions \
     -H "Authorization: Bearer $LITELLM_VIRTUAL_KEY" \
     -H "Content-Type: application/json" \
     -d '{
       "model": "glm-latest",
       "messages": [{"role": "user", "content": "Say pong in one word."}]
     }'
   

​

Use Nexus Auto through a gateway

The gateway sends the auto model ID and your Fireworks API key to Fireworks. Nexus CLI is not required. For LiteLLM entries, the Portkey model format, and a test request, see Nexus Auto.

​

Provide closed-model credentials

auto chooses from the models your credentials allow:

• With an Anthropic credential, it can also use Anthropic models.
• With an OpenAI credential, it can also use OpenAI models.
• With neither, every turn uses Fireworks open models.

An Amazon Bedrock Provider Key covers a closed model you mapped there.

Provider keys can be stored once in the gateway or connected as Provider Keys, so clients and harnesses don’t have to send them:

Setup	Where the provider key lives	Clients send
Provider Keys	Your Fireworks account	Only their gateway key
Keys in the gateway config	The gateway’s config or secret store	Only their gateway key
Keys from each client	Each client or harness	Their gateway key and a provider-key header

Provider Keys

An account admin connects the required Provider Keys once from Settings. The gateway stores only the Fireworks key. No provider credential passes through the gateway.

Every Nexus plan includes Provider Keys. You can also use your own key: save it with nexus auto, or send it as a request header.

curl http://localhost:4000/chat/completions \
  -H "Authorization: Bearer $LITELLM_VIRTUAL_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "auto",
    "messages": [{"role": "user", "content": "Say pong in one word."}]
  }'

Keys in the gateway config

Use this option when Provider Keys isn’t turned on for your account. The gateway adds the provider-key headers to every request for Nexus Auto, so clients and harnesses send only their gateway key.

In LiteLLM, add extra_headers to the auto entry. os.environ/ reads the key from the proxy’s environment:

config.yaml

model_list:
  - model_name: auto
    litellm_params:
      model: fireworks_ai/accounts/fireworks/routers/auto
      api_key: os.environ/FIREWORKS_API_KEY
      api_base: https://api.fireworks.ai/inference/v1
      extra_headers:
        x-anthropic-api-key: os.environ/ANTHROPIC_API_KEY

To give Nexus Auto an OpenAI key too, add x-openai-api-key: os.environ/OPENAI_API_KEY under extra_headers.

For Portkey and other gateways, set the same headers as custom headers on the Fireworks provider or integration.

Keys from each client

Use this option when each team or user should call closed models with their own provider key. The client sends x-anthropic-api-key, x-openai-api-key, or both to the gateway.

LiteLLM does not forward client headers by default. Enable header forwarding in config.yaml:

general_settings:
  forward_client_headers_to_llm_api: true

curl http://localhost:4000/chat/completions \
  -H "Authorization: Bearer $LITELLM_VIRTUAL_KEY" \
  -H "Content-Type: application/json" \
  -H "x-anthropic-api-key: $ANTHROPIC_API_KEY" \
  -d '{
    "model": "auto",
    "messages": [{"role": "user", "content": "Say pong in one word."}]
  }'

Gateways often log request headers. If provider keys travel as headers, check that the gateway redacts them in its logs. This applies whether the gateway adds the headers or clients send them. Provider Keys keeps provider credentials out of the gateway entirely.

LiteLLM returns your model_name in the response model field, not the model that served the turn. To confirm that closed models are reachable, send a request directly to Fireworks as shown in Check which model served a turn. For direct API calls, see APIs and SDKs.