In addition to the predefined set of models already available on Fireworks, you can also upload your own custom models. To upload a custom LoRA addon, see importing fine-tuned models.

​
Requirements

Fireworks currently supports the following model architectures:

The model files you will need to provide depend on the model architecture. In general, you will need the following files:

  • Model configuration: config.json.

    Fireworks does not support the quantization_config option in config.json.

  • Model weights, in one of the following formats:

    • *.safetensors
    • *.bin

  • Weights index:*.index.json

  • Tokenizer file(s), e.g.

    • tokenizer.model
    • tokenizer.json
    • tokenizer_config.json

If the requisite files are not present, model deployment may fail.

​
Enabling chat completions

To enable the chat completions API for your custom base model, ensure your tokenizer_config.json contains a chat_template field. See the Hugging Face guide on Templates for Chat Models for details.

​
Uploading the model locally (firectl)

To upload a custom base model, run the following command.

firectl create model <MODEL_ID> /path/to/files/

​
Uploading models from S3 buckets (firectl)

For larger models, you can upload directly from an Amazon S3 bucket, which provides a faster transfer process than uploading from local files.

To upload a model directly from an S3 bucket, run the following command.

firectl create model <MODEL_ID> s3://<BUCKET_NAME>/<PATH_TO_MODEL>/ --aws-access-key-id <ACCESS_KEY_ID> --aws-secret-access-key <SECRET_ACCESS_KEY>

See the AWS documentation for how to generate an access key ID and secret access key pair.

Ensure the IAM user has read access to the S3 bucket containing the model.

​
Uploading via REST API (python)

For more programmatic control, you can use the Fireworks REST API to upload your custom models. This involves a four-step process:

  1. Create a model object: This creates a reference to your model in the Fireworks system.
  2. Get signed upload URLs: For each of your model files, you’ll get a unique URL to upload to.
  3. Upload files: Upload each file to its corresponding signed URL.
  4. Validate the upload: This tells Fireworks to verify the integrity of the uploaded files and make the model available for deployment.

Here’s a Python script demonstrating this process:

import os
import requests
import json

# Make sure to set your API key as an environment variable
API_KEY = os.environ.get("FIREWORKS_API_KEY")
ACCOUNT_ID = "my-account" # Replace with your account ID
MODEL_ID = "my-custom-model" # Replace with your desired model ID
MODEL_PATH = "/path/to/your/model/files/" # Path to your local model files

BASE_URL = "https://api.fireworks.ai/api/v1"
HEADERS = {
    "Authorization": f"Bearer {API_KEY}",
    "Content-Type": "application/json",
}

def create_model():
    """Step 1: Create a model object."""
    print("Creating model...")
    url = f"{BASE_URL}/accounts/{ACCOUNT_ID}/models"
    payload = {
        "modelId": MODEL_ID,
        "model": {
            # This assumes a HuggingFace base model.
            # Change 'kind' for different model types.
            "kind": "HF_BASE_MODEL",
            "baseModelDetails": {
                "checkpointFormat": "HUGGINGFACE",
                "worldSize": 1,
            },
        },
    }
    response = requests.post(url, headers=HEADERS, data=json.dumps(payload))
    response.raise_for_status()
    print("Model created successfully.")
    return response.json()

def get_upload_urls():
    """Step 2: Get signed upload URLs."""
    print("Getting upload URLs...")
    filename_to_size = {
        f: os.path.getsize(os.path.join(MODEL_PATH, f))
        for f in os.listdir(MODEL_PATH)
        if os.path.isfile(os.path.join(MODEL_PATH, f))
    }

    url = f"{BASE_URL}/accounts/{ACCOUNT_ID}/models/{MODEL_ID}:getUploadEndpoint"
    payload = {"filenameToSize": filename_to_size}
    response = requests.post(url, headers=HEADERS, data=json.dumps(payload))
    response.raise_for_status()
    print("Upload URLs received.")
    return response.json()["filenameToSignedUrls"]

def upload_files(upload_urls):
    """Step 3: Upload model files."""
    print("Uploading files...")
    for filename, url in upload_urls.items():
        print(f"  Uploading {filename}...")
        file_path = os.path.join(MODEL_PATH, filename)
        with open(file_path, "rb") as f:
            # Note: The upload URL is pre-signed, so we don't need authorization headers
            response = requests.put(url, data=f)
            response.raise_for_status()
    print("All files uploaded successfully.")

def validate_upload():
    """Step 4: Validate the upload."""
    print("Validating model upload...")
    url = f"{BASE_URL}/accounts/{ACCOUNT_ID}/models/{MODEL_ID}:validateUpload"
    # No payload is needed for this request
    response = requests.post(url, headers=HEADERS)
    response.raise_for_status()
    print("Model validation successful. Your model is ready to be deployed.")

if __name__ == "__main__":
    create_model()
    urls = get_upload_urls()
    upload_files(urls)
    validate_upload()

​
Deploying

A model cannot be used for inference until it is deployed. See the Deploying models guide to deploy the model.

​
Publishing

By default, all models you create are only visible to and deployable by users within your account. To publish a model so anyone with a Fireworks account can deploy it, you can create it with the --public flag. This will allow it to show up in public model lists.

firectl create model <MODEL_ID> /path/to/files --public

To unpublish the model, just run

update
firectl update model <MODEL_ID> --public=false