> ## Documentation Index > Fetch the complete documentation index at: https://fireworks.ai/docs/llms.txt > Use this file to discover all available pages before exploring further. # Evaluating Trained Models > Evaluate a trained Fireworks model by sampling its serverless checkpoints with the Training SDK, by loading a promoted LoRA onto serverless, or with a preemptible deployment. After training, evaluate your model before you serve it in production. Choose an evaluation path based on the artifact you have. > [!WARNING] > > Serverless checkpoint sampling and preemptible deployments are for **training evals and other non-production workloads only**. Do not send production or latency-sensitive traffic through them. For production serving after eval, create an on-demand deployment. See [Deploying Trained Models](https://fireworks.ai/docs/fine-tuning/deploying-loras.md). LoRAs trained on [select base models](https://fireworks.ai/docs/serverless/specialized-model-inference/overview.md#supported-models-and-limits) can also be served on serverless with [Specialized Model Inference](https://fireworks.ai/docs/serverless/specialized-model-inference/overview.md). ## Which approach to use Use the path that matches your current training stage: * **You have a serverless sampler checkpoint:** sample it with the Training SDK, during or after the run. * **You have a promoted LoRA on a [supported base model](https://fireworks.ai/docs/serverless/specialized-model-inference/overview.md#supported-models-and-limits):** load it onto serverless with [Specialized Model Inference](#serverless-lora-inference) and pay per token. * **You have a promoted model ID on any other base model:** use a preemptible deployment. This works regardless of whether the LoRA was trained with serverless, dedicated, or managed training, or imported. * **Your dedicated training run is still active:** use its inference deployment and refresh it from sampler snapshots. See [Dedicated training and sampling](https://fireworks.ai/docs/fine-tuning/training-api/dedicated.md#training-and-sampling). | | **Serverless checkpoint sampling** | **Serverless LoRA inference** | **Preemptible deployment** | | - | - | - | - | | **Best for** | Quick checks of serverless checkpoints, during or after the run | Evaluating a promoted LoRA on a supported base model, billed per token | Evaluating a promoted model without reserving dedicated GPUs | | **Training path** | Serverless Training API only | Any, including imported adapters | Any training path | | **Artifact** | Sampler checkpoint from `save_weights_for_sampler` | Promoted LoRA (`accounts//models/`) | Promoted model (`accounts//models/`) | | **API** | Training SDK `sampler.sample()` | Chat Completions | Chat Completions or Fireworks SDK | | **Needs an active training session?** | No | No | No | ## Serverless checkpoint sampling Sample a sampler checkpoint from [serverless training](https://fireworks.ai/docs/fine-tuning/training-api/serverless.md) while its base model is available in the [serverless pool](https://fireworks.ai/docs/fine-tuning/models.md). The training session does not need to be active: any process can open a sampler for a checkpoint your account saved. This does not apply to an adapter produced by dedicated training. A sampler checkpoint has no chat endpoint. Open a **sampling client** bound to a sampler checkpoint. `sampler.sample()` generates completions from that checkpoint, which you score with your own metric. Use the **sampler checkpoint** returned by `save_weights_for_sampler`, not a promoted model resource (`accounts//models/`). Promoted models are served through deployments or Specialized Model Inference and cannot be passed to `create_sampling_client`. The following save-and-sample sequence follows [Train, checkpoint, and sample](https://fireworks.ai/docs/fine-tuning/training-api/serverless.md#train-checkpoint-and-sample) on the serverless training page. Set up `prompt`, `tokenizer`, and `params` as you do for training rollouts. ```python theme={null} snapshot = training_client.save_weights_for_sampler("eval").result().path sampler = service.create_sampling_client(model_path=snapshot, tokenizer=tokenizer) try: result = sampler.sample( prompt=prompt, num_samples=1, sampling_params=params, ).result() for seq in result.sequences or []: tokens = list(seq.tokens or []) completion = get_text_content(renderer.parse_response(tokens)[0]) # Score `completion` against your held-out label or grader. finally: sampler.close() ``` `sampler.sample(...)` is the evaluation call. Repeat it over held-out prompts, score each completion with your evaluation metric, then close the sampler. The cookbook's [DeepMath serverless example](https://github.com/fw-ai-external/cookbook/blob/main/training/examples/rl/deepmath/train_deepmath_serverless.py) shows held-out accuracy evaluation with a math reward. For checkpoint and promotion details, see [Saving and loading checkpoints](https://fireworks.ai/docs/fine-tuning/training-api/serverless.md#saving-and-loading-checkpoints) on the serverless training page. ### After the run ends You do not need a training client to sample a checkpoint: open a sampler from a fresh service client with the checkpoint path that `save_weights_for_sampler` returned. See [Sampling without a training client](https://fireworks.ai/docs/fine-tuning/training-api/serverless.md#sampling-without-a-training-client) on the serverless training page, which also covers running evaluation across worker processes. > [!NOTE] > > Promote checkpoints you want to keep long term or serve through the chat completions API, then evaluate the promoted model on [serverless](#serverless-lora-inference) if its base model is supported, or with a [preemptible deployment](#preemptible-deployment). ## Serverless LoRA inference If your promoted LoRA was trained on a [supported base model](https://fireworks.ai/docs/serverless/specialized-model-inference/overview.md#supported-models-and-limits), you can load it onto Fireworks' shared serverless pool and call it through the Chat Completions API like any serverless model. You don't create a deployment, and you pay per token at the base model's price. This works for adapters from any training path, as well as imported adapters, as long as the base model is supported and the LoRA rank is within its limit. See [Specialized Model Inference on Serverless](https://fireworks.ai/docs/serverless/specialized-model-inference/overview.md) for supported models, limits, and how to load, call, and unload an adapter. ## Preemptible deployment If you want to evaluate a promoted model without holding dedicated on-demand capacity, create a **preemptible deployment**. It borrows idle reserved GPU capacity instead of reserving GPUs exclusively for you. For LoRA, this includes adapters trained with serverless, dedicated, or managed training, as well as imported adapters. Point the deployment at your promoted fine-tuned model ID, not the base model. It can be reclaimed (preempted) at any time. Fireworks does not guarantee how many GPUs are available or how long the deployment stays up, but in practice it typically lasts long enough to finish a training eval. ### How it works Passing `--preemptible` to `firectl deployment create` opts the deployment into capacity borrowing: * The deployment runs on reserved nodes that are currently idle. * When the capacity owner needs those GPUs back, your deployment can be preempted. Because it borrows idle capacity, you do not need to hold dedicated on-demand capacity for the duration of the eval. ### Requirements `--preemptible` takes effect only on **firectl 1.7.26 or newer**. Check with `firectl version`. The flag is not present in older builds. Upgrade if you are below 1.7.26. ### Behavior * **Training evals only.** A preemptible deployment can be preempted mid-request. Treat disappearance as a normal outcome, not an error. * **`--preemptible` is immutable.** It is set at create time and cannot be toggled on or off afterward. To change it, delete the deployment and create a new one. * **Clean up when done.** Delete the deployment after your eval so you stop holding the borrowed capacity. ### Create the deployment ```bash theme={null} firectl deployment create accounts//models/ \ --deployment-id \ --display-name \ --deployment-shape default \ --min-replica-count 1 \ --max-replica-count 1 \ --preemptible ``` Replace the placeholders: * ``: the trained model to eval, not a base model. * `` / ``: a name of your choice for the eval deployment. `--deployment-shape default` has Fireworks pick a validated shape (for fine-tuned models, matched against the model's base model). To pick a specific shape, see [Deployment shapes](https://fireworks.ai/docs/guides/ondemand-deployments.md#deployment-shapes). ### Worked example ```bash theme={null} firectl deployment create accounts//models/ \ -a \ --deployment-id -eval \ --display-name -eval \ --deployment-shape default \ --min-replica-count 1 \ --max-replica-count 1 \ --preemptible ``` ### Run the eval and tear down Check that the deployment is ready, then send eval requests to the trained model: ```bash theme={null} firectl deployment get -a ``` ```bash theme={null} curl https://api.fireworks.ai/inference/v1/chat/completions \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $FIREWORKS_API_KEY" \ -d '{ "model": "accounts//models/", "messages": [ { "role": "user", "content": "Hello!" } ] }' ``` When the evaluation is finished, delete the deployment to release the borrowed capacity: ```bash theme={null} firectl deployment delete -a ``` For more on preemptible capacity and guarantees, see [Preemptible deployments](https://fireworks.ai/docs/guides/ondemand-deployments.md#preemptible-deployments-eval-batch) in the on-demand deployments guide. ## Next steps > [card] [Deploying Trained Models](https://fireworks.ai/docs/fine-tuning/deploying-loras.md) > > Live merge or multi-LoRA for production serving > [card] [Specialized Model Inference](https://fireworks.ai/docs/serverless/specialized-model-inference/overview.md) > > Serve LoRAs on supported base models on serverless This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.