> ## Documentation Index > Fetch the complete documentation index at: https://fireworks.ai/docs/llms.txt > Use this file to discover all available pages before exploring further. # Serverless Rate Limits > How per-model adaptive rate limits work on Fireworks serverless inference: bounds, adjustment logic, and how limits grow and shrink with your recent usage of each model. When using Serverless, you may experience `429 Too Many Requests` or `503 Service Overloaded`. To avoid 429s, you need to stay below our adaptive rate limits. To reduce the likelihood of 503s, you can upgrade to [Priority tier](https://fireworks.ai/docs/serverless/serverless-modes.md). **Rate limits are per account and per model — there is no single account-wide token pool.** Each model has its own TPM limits, and each model's adaptive limit grows, shrinks, and resets based only on your usage of that model. Using one model does not consume another model's limit, and hitting a limit on one model does not rate limit your traffic to other models. ## What are your rate limits? There are three metrics we use to rate limit each account, separately for each model: * **Total Prompt TPM** — input tokens per minute (cached + uncached). * **Uncached Prompt TPM** — uncached input tokens per minute. * **Generated TPM** — output tokens per minute. All limits are measured and enforced in **tokens per minute (TPM)**, per model. For example, exhausting your Total Prompt TPM on one model returns `429`s for that model only — your other models keep their own, unaffected Total Prompt TPM limits. Base adaptive rate limit ceilings depend on the model's total parameter count. Smaller models get higher base ceilings:
Tier Total parameters Total\ Prompt TPM Uncached\ Prompt TPM Generated\ TPM
**Small** \< 800B 216M 54M 2.16M
**Medium** 800B – \< 1.6T 43.2M 10.8M 432k
**Large** ≥ 1.6T 21.6M 5.4M 216k
Fast, Priority, and US-only variants of a model share the same tier and ceilings as the base model. Ultra-fast (Preview) has lower rate limits than the standard model (see below). Models without a known parameter count use **Large** ceilings. ### Models by tier | Tier | Models | | - | - | | **Small** | DeepSeek V4.1 Flash, GLM 5.3, GLM 5.3 Flash, MiniMax M3, NVIDIA Nemotron 3 Ultra (Preview), OpenAI GPT OSS 120B, NVIDIA Nemotron 3.5 Lightning 30B A3B, Rune 26B-A4B | | **Medium** | Muse Spark Open, DeepSeek V4.1 Flash 1017 Ultra-fast (Preview) | | **Large** | Ember-1, Kimi K3, Kimi K3.1 Pro, Qwen 3.8 Max | Models not listed here are tiered by total parameter count using the thresholds above. Based on your usage, your adaptive limits will grow and shrink within these base ceilings. Eligible accounts may also receive temporary capacity above the base ceiling through [Elastic Burst](#elastic-burst). If your traffic ramps up too quickly, you will get 429s. ![kimi-k2p6 usage and rate limits](https://mintcdn.com/fireworksai-docs/irNNOrw0rhhJ50GA/images/serverless/ratelimit-example.png?fit=max\&auto=format\&n=irNNOrw0rhhJ50GA\&q=85\&s=edbe8a1a601cae9dc1365e1b958d3810) Historical limit samples for each model are in the [Serverless dashboard](https://app.fireworks.ai/dashboard/serverless). See [Checking your current limit](#checking-your-current-limit). Adaptive rate limits have an upper and lower bound. A higher account [Spending Tier](https://fireworks.ai/docs/guides/quotas_usage/account-quotas.md#spending-tiers) correlates with higher upper bound rate limits; **enterprise accounts** get higher upper bounds automatically. ## How do adaptive limits change? Adaptive rate limiting is also **per model**: each model's limit adapts independently, based only on your recent usage of that model. Ramping up one model does not raise another model's limit, and going idle on one model does not shrink another model's limit. Your limit for each model starts at a default value and grows in steps as you use it. If your recent one-minute usage is above 50% of your current limit, the limit increases by 25%, and no more than once every 15 minutes, until it reaches your base ceiling. If you stop using it, the limit shrinks slowly and eventually resets. [Elastic Burst](#elastic-burst) can move the current limit above that ceiling, and back toward it, independently of the schedule below. Each of the three metrics (Total Prompt TPM, Uncached Prompt TPM, and Generated TPM) has its own standard adaptive limit and changes on its own. Heavy usage on one metric does not change the other standard limits. Under Elastic Burst, high usage on one of these metrics can reduce burst capacity on the others. ### Your base ceiling The base ceiling is the highest your standard adaptive limit can grow without Elastic Burst. Two things set it: * **Model size** picks the row in the [table above](#model-size-tiers). * **Your [spending tier](https://fireworks.ai/docs/guides/quotas_usage/account-quotas.md#spending-tiers)** sets how much of that row you can reach. The table shows the maximum ceilings, available at Tier 3 and above. Tiers below Tier 3 have lower ceilings. ### Elastic Burst Elastic Burst can temporarily raise an eligible account's current limit above its base ceiling when spare capacity is available. It applies to Total Prompt TPM, Uncached Prompt TPM, and Generated TPM. For eligible accounts and models, Elastic Burst applies automatically to the existing Serverless endpoint; no request or client changes are required. That extra capacity is not guaranteed, and it can change independently of the standard schedule below. If your current limit drops, pace traffic against the new limit. For `429` responses, honor the `Retry-After` header; if it is absent, retry with exponential backoff and jitter. Elastic Burst does not change your base ceiling or any [Reserved Throughput](https://fireworks.ai/docs/serverless/reserved-throughput.md). Elastic Burst has no separate enablement fee or burst premium. Usage is billed at your applicable Serverless token rates. To opt out, contact your Fireworks account representative. The opt-out is account-wide and applies across all models for Total Prompt TPM, Uncached Prompt TPM, and Generated TPM. ### When limits go up, down, or reset | Change | When it happens | Amount | | - | - | - | | **Increase** | Your recent usage is above 50% of your current limit. Increases happen at most once every 15 minutes. | Up 25%, capped at your base ceiling | | **Decrease** | Your highest sampled 15-minute usage over the past 3 days stayed below 33% of your current limit, and at least 24 hours have passed since the limit last changed. | Down 20%, never below the lower bound | | **Reset** | You send no traffic to the model for more than 72 hours. | Back to the starting default | | **Elastic Burst** | Your usage is above 50% of your current limit and spare capacity is available. It moves back toward your base ceiling when burst headroom goes unused or capacity is needed elsewhere | Increases and decreases vary with available capacity, burst won't affect your base ceiling | A few details: * A decrease is skipped if your recent usage would be at least 50% of the lower limit. This keeps the limit from dropping and then immediately climbing back up. * With [Reserved Throughput](https://fireworks.ai/docs/serverless/reserved-throughput.md), the increase, decrease, and reset thresholds apply to usage above your reservation and to the adaptive portion of your limit. Usage within the reservation does not count toward the thresholds. * Resets do not apply to accounts with a custom limit or a reservation for that model. * Elastic Burst example: if your current limit is 100M TPM, Elastic Burst can raise it once your usage is above 50M TPM and spare capacity is available. ### How slowly limits shrink The steps below follow the standard **Decrease** row. Elastic Burst capacity can leave without waiting for these daily steps. Say your Total Prompt TPM limit is 10,000,000 and your traffic drops to a steady 1,000,000 tokens per minute: | Day | Limit (tokens per minute) | What happens | | - | - | - | | 0 to 3 | 10,000,000 | No change. Your earlier, heavier traffic is still in the 3 day window. | | 3 | 8,000,000 | First decrease, once the heavy traffic ages out of the window. | | 4 | 6,400,000 | | | 5 | 5,120,000 | | | 6 | 4,096,000 | | | 7 | 3,276,800 | | | 8 | 2,621,440 | Last decrease. 1,000,000 is now more than one third of the limit. | If you stop sending traffic completely, you don't go through these steps. After more than 72 hours with no traffic, the limit resets to the starting default. ### Example: ramping up Say your current Total Prompt TPM limit is 1,000,000. 1. Send more than 500,000 but less than 1,000,000 prompt tokens per minute. This keeps you above half your limit without hitting 429s. 2. More than 15 minutes later, your limit rises to 1,250,000. 3. Raise your traffic to match, staying between 625,000 and 1,250,000. 4. Repeat until your limit stops rising. At this pace, you reach your base ceiling in a few hours. If you are using more than half your limit and it has stopped rising after an increase interval, the standard adaptive limit is at the base ceiling for your spending tier. Moving to a higher spending tier, up to Tier 3, raises the base ceiling, and that limit keeps growing in the same 25% steps. Elastic Burst does not follow these steps. A short traffic spike does not lock in a higher limit. If you go 72 hours without traffic, the limit resets to the default. ### Checking your current limit The [Serverless dashboard](https://app.fireworks.ai/dashboard/serverless) shows historical limit samples, not an instantaneous reading. A Serverless response can include the latest known effective limit in these headers when that limit is known. The value can lag a recent change. * `X-Ratelimit-Limit-Tokens-Prompt` for Total Prompt TPM * `X-Ratelimit-Limit-Tokens-Uncached-Prompt` for Uncached Prompt TPM * `X-Ratelimit-Limit-Tokens-Generated` for Generated TPM These headers are not present on every response. Header values are tokens per minute. Elastic Burst can put these three above the published base ceilings. `X-Ratelimit-Limit-Tokens-Cache-Adjusted-Prompt` is a separate limit and does not include Elastic Burst. See [response headers](https://fireworks.ai/docs/serverless/overview.md#response-headers). ## FAQ > [accordion] Am I guaranteed successful responses up to my rate limit? > > **No.** Staying within your rate limits does not guarantee that every request succeeds. When a deployment is busy, your traffic can still be **load shed**, and those responses are **`503 Service Overloaded`**. To **decrease the chance** of being load shed, you can use [Priority tier](https://fireworks.ai/docs/serverless/serverless-modes.md), which is prioritized during high load. > [accordion] How are rate limits scoped? > > Rate limits — including adaptive rate limiting — are scoped **per account** and **per model**. Each model has its own TPM limits and its own adaptive limit, which grows, shrinks, and resets based only on your usage of that model. Traffic to one model does not consume, raise, or lower another model's limits. **Fast** and **regular** model variants have **separate** limits. **Priority tier** and **regular** requests share the **same** rate limits for a given model. > [accordion] How is my model's ceiling tier determined? > > Ceiling tiers are based on the model's **total parameter count**: **Small** (\< 800B), **Medium** (800B – \< 1.6T), or **Large** (≥ 1.6T). See [Model size tiers](#model-size-tiers) for the ceiling values and [Models by tier](#models-by-tier) for where each Serverless model falls. > [accordion] What should I do first when I see 429s? > > Honor the response's `Retry-After` header. If it is absent, retry with **exponential backoff and jitter**. > [accordion] How do I get higher limits sooner? > > Reach out to [inquiries@fireworks.ai](mailto:inquiries@fireworks.ai) for a custom solution if either of these applies: > > * **You need higher than the defaults from day one.** Your launch traffic exceeds the starting limit and you can't wait for the adaptive ramp. > * **You're ramping past the highest base ceiling.** You are already at the highest account Spending Tier and the standard adaptive limit is not growing. This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.