Chat Completions

The Chat Completion API allows you to generate conversational responses from Nextbit's language models. Our API is fully compatible with OpenAI's chat completions format.

The chat completion model

The chat completion model contains all the information about your conversation, including messages, system prompts, and generation parameters.

Properties

Name
model
Type
string
Description
The ID of the model to use. See available models for the complete list with pricing.
Name
messages
Type
array
Description
An array of message objects representing the conversation history. Each message should have a role (system, user, or assistant) and content.
Name
temperature
Type
number
Description
Controls randomness in the output. Higher values (e.g., 1.5) make output more random, lower values (e.g., 0.5) make it more focused. Range: 0-2, default: 1.
Name
max_tokens
Type
integer
Description
The maximum number of tokens to generate in the completion.
Name
stream
Type
boolean
Description
If set to true, partial message deltas will be sent as server-sent events. Default: false.
Name
top_p
Type
number
Description
An alternative to temperature sampling. Range: 0-1, default: 1.
Name
presence_penalty
Type
number
Description
Number between -2.0 and 2.0. Positive values penalize new tokens based on whether they appear in the text so far.
Name
frequency_penalty
Type
number
Description
Number between -2.0 and 2.0. Positive values penalize new tokens based on their existing frequency in the text.

POST/v1/chat/completions

Create chat completion

This endpoint allows you to create a chat completion using one of our available models.

Required attributes

Name
model
Type
string
Description
The model ID to use for completion.
Name
messages
Type
array
Description
Array of message objects with role and content.

Optional attributes

Name
temperature
Type
number
Description
Sampling temperature (0-2). Default: 1.
Name
max_tokens
Type
integer
Description
Maximum tokens to generate.
Name
stream
Type
boolean
Description
Enable streaming responses. Default: false.

Request

POST

/v1/chat/completions

from openai import OpenAI

client = OpenAI(
    base_url="https://api.nextbit256.com/v1",
    api_key="your-api-key"
)

response = client.chat.completions.create(
    model="llama3.3:70b",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "What is the capital of Spain?"}
    ]
)

print(response.choices[0].message.content)

Response

{
  "id": "chatcmpl-6ff64705f50d43cbbab7cd03250e9382",
  "object": "chat.completion",
  "created": 1742470129,
  "model": "llama3.3:70b",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "The capital of Spain is Madrid."
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 20,
    "completion_tokens": 8,
    "total_tokens": 28
  }
}

POST/v1/chat/completions

Streaming

For a better user experience, you can stream the model's response token by token. Set stream: true in your request to enable streaming.

The response will be sent as server-sent events (SSE), with each event containing a chunk of the completion.

Request

POST

/v1/chat/completions

from openai import OpenAI

client = OpenAI(
    base_url="https://api.nextbit256.com/v1",
    api_key="your-api-key"
)

stream = client.chat.completions.create(
    model="llama3.3:70b",
    messages=[
        {"role": "user", "content": "Tell me about San Francisco"}
    ],
    stream=True
)

for chunk in stream:
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)