> ## Documentation Index
> Fetch the complete documentation index at: https://fireworks.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Serverless Inference Overview

> How Serverless inference works on Fireworks: serverless modes, per-token billing, request headers, prompt caching, model lifecycle, and when to pick Serverless.

## What is Serverless Inference

Serverless Inference is multi-tenant inference for popular open models running on Fireworks-managed infrastructure. You point your client at `api.fireworks.ai`, send tokens, and pay only for what you use — no GPUs to size, no autoscaler to tune, no cold starts to wait through. Models eligible for Serverless carry the **Serverless** tag in the [model library](https://fireworks.ai/models). To make your first call, see the [Serverless quickstart](https://fireworks.ai/docs/getting-started/quickstart.md).

## Serverless products at a glance

Four modes run on the same Serverless framework. They route and price differently, and share the same rate-limit policy, except for Preview offerings such as Ultra-fast, which have their own limits (see [Serverless rate limits](https://fireworks.ai/docs/serverless/rate-limits.md#models-by-tier)):

* **Standard** — the default mode. No `service_tier` parameter needed.
* **Priority** — higher reliability during peak periods. Opt in by setting `service_tier: "priority"` on chat completions. Priced at a premium.
* **Fast** — high-speed deployments for latency-sensitive workloads. Selected by switching the `model` ID to a Fast variant (for example, `accounts/fireworks/routers/kimi-k3-fast`).
* **Ultra-fast (Preview)** — very high-speed mode for highly latency-sensitive workloads, aiming for 6-8x generation speed. Selected by switching the `model` ID to an Ultra-fast variant (for example, `accounts/fireworks/routers/deepseek-v4p1-flash-1017-ultrafast`). Priced at a premium.

For usage examples and the full list of supported models, see [Serverless Modes](https://fireworks.ai/docs/serverless/serverless-modes.md). For pricing by mode, see [Serverless pricing](https://fireworks.ai/docs/serverless/pricing.md).

## Billing

Serverless is priced per token. Three dimensions are billed:

* **Input tokens** — what you send to the model.
* **Cached input tokens** — input tokens served from prompt cache, discounted (default 50% of input on text and vision models, unless a model lists a different cached rate).
* **Generated tokens** — what the model produces.

Decision models such as [Rune 26B-A4B](https://app.fireworks.ai/models/fireworks/rune) are billed on input only: they return calibrated probabilities for typed questions and never generate tokens.

Other things to know:

* The `usage` object in each response is the source of truth for what was billed (`prompt_tokens`, `completion_tokens`, `total_tokens`).
* **Batch inference** is billed at 50% of standard Serverless rates on both input and output. See [Batch inference](https://fireworks.ai/docs/guides/batch-inference.md).
* Your **spend tier** influences Serverless capacity caps. Your monthly spend limit is a separate account control. Higher spend tiers unlock higher TPM upper bounds. See [Account quotas](https://fireworks.ai/docs/guides/quotas_usage/account-quotas.md) and [Serverless rate limits](https://fireworks.ai/docs/serverless/rate-limits.md).

## Request and response headers

Headers a Serverless caller will set or read.

### Request headers

| Header | Notes |
| - | - |
| `Authorization: Bearer $FIREWORKS_API_KEY` | Required for all requests. |
| `x-session-affinity` | Optional sticky-routing key. Pin repeated requests to the same replica to maximize prompt-cache hit rate. See [Prompt caching](https://fireworks.ai/docs/guides/prompt-caching.md). |

### Response headers

Fireworks sets the following on Serverless inference responses. The rate-limit headers are included only when that limit is known:

| Header | What it tells you |
| - | - |
| `fireworks-prompt-tokens` | Input tokens for the request. |
| `fireworks-cached-prompt-tokens` | Cached portion of the input. See [Prompt caching](https://fireworks.ai/docs/guides/prompt-caching.md#monitoring). |
| `X-Ratelimit-Limit-Tokens-Prompt` | Latest known Total Prompt TPM (tokens per minute) limit. |
| `X-Ratelimit-Limit-Tokens-Uncached-Prompt` | Latest known Uncached Prompt TPM limit. |
| `X-Ratelimit-Limit-Tokens-Generated` | Latest known Generated TPM limit. |
| `X-Ratelimit-Limit-Tokens-Cache-Adjusted-Prompt` | Latest known cache-adjusted prompt limit. This is separate from Uncached Prompt TPM. |

`X-Ratelimit-Limit-Tokens-Prompt`, `X-Ratelimit-Limit-Tokens-Uncached-Prompt`, and `X-Ratelimit-Limit-Tokens-Generated` show the latest known effective limit in tokens per minute, which can lag a recent change. [Elastic Burst](https://fireworks.ai/docs/serverless/rate-limits.md#elastic-burst) can raise those three above the [published base ceilings](https://fireworks.ai/docs/serverless/rate-limits.md#model-size-tiers). It does not apply to `X-Ratelimit-Limit-Tokens-Cache-Adjusted-Prompt`.

> [!TIP]
>
> Streaming responses don't carry per-request perf headers. To get the same metrics in the streaming response body, set the `perf_metrics_in_response` parameter on the request. See [Querying text models](https://fireworks.ai/docs/guides/querying-text-models.md#usage--performance-tracking).

## Prompt caching

Prompt caching is on by default for every Serverless model. Cached input tokens are billed at the discounted rate (default 50% of input). Caching is replica-local, so to maximize hit rate you should route repeated prompts to the same replica — pass a stable identifier in `x-session-affinity` (or in the OpenAI `user` field) for each user or session whose prompts share a prefix.

For the full guide, including how to structure prompts for cache hits and how to read cache metrics, see [Prompt caching](https://fireworks.ai/docs/guides/prompt-caching.md).

## Serverless model lifecycle

Serverless models are managed by the Fireworks team and may be updated or deprecated as new models are released. We provide **at least 2 weeks advance notice** before removing any model, with longer notice periods for popular models based on usage.

For production workloads requiring long-term model stability, we recommend [on-demand deployments](https://fireworks.ai/docs/guides/ondemand-deployments.md), which give you full control over model versions and updates.

## Serverless vs On-demand

| When Serverless fits | When On-demand fits |
| - | - |
| Pay per token, only for what you use | Pay per GPU-hour for dedicated capacity |
| You're using popular base models that Fireworks already hosts, or LoRAs on [select base models](https://fireworks.ai/docs/serverless/specialized-model-inference/overview.md#supported-models-and-limits) | You're running custom base models, or LoRAs on base models that serverless doesn't support |
| You don't want to manage scaling, replicas, or hardware sizing | You have custom latency requirements and want control over hardware and replicas |

For dedicated infrastructure, see [On-demand deployments](https://fireworks.ai/docs/guides/ondemand-deployments.md).

## Next steps

> [card] [Serverless quickstart](https://fireworks.ai/docs/getting-started/quickstart.md)
>
> Make your first Serverless API call.

> [card] [Serverless Modes](https://fireworks.ai/docs/serverless/serverless-modes.md)
>
> Higher-reliability and higher-speed modes.

> [card] [US-only Serverless](https://fireworks.ai/docs/serverless/us-only-serverless.md)
>
> Dedicated US routers for inference residency.

> [card] [Pricing](https://fireworks.ai/docs/serverless/pricing.md)
>
> Per-token rates for text, vision, embeddings, and Priority.

> [card] [Reserved Throughput](https://fireworks.ai/docs/serverless/reserved-throughput.md)
>
> SLA-backed throughput that adds to your adaptive rate limits.

> [card] [Rate limits](https://fireworks.ai/docs/serverless/rate-limits.md)
>
> Adaptive TPM bounds and how the limit ramps with usage.

> [card] [Prompt caching](https://fireworks.ai/docs/guides/prompt-caching.md)
>
> How caching works and how to maximize hit rate.

> [card] [On-demand deployments](https://fireworks.ai/docs/guides/ondemand-deployments.md)
>
> Dedicated GPUs for predictable throughput and custom models.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.