Chat Completions

Generate chat completions using the OpenAI-compatible API.

Endpoint

POST /v1/chat/completions

Request Body

{
  "messages": [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Hello!"}
  ],
  "max_tokens": 100,
  "temperature": 0.7,
  "stream": false
}

Parameters

Parameter	Type	Required	Description
messages	array	Yes	Conversation messages
max_tokens	integer	No	Maximum tokens to generate
temperature	float	No	Sampling temperature (0-2)
top_p	float	No	Nucleus sampling (0-1)
stream	boolean	No	Enable streaming response
stop	string/array	No	Stop sequences

Message Format

{
  "role": "user",
  "content": "Your message here"
}

Supported roles: system, user, assistant

Response

{
  "id": "chatcmpl-abc123",
  "object": "chat.completion",
  "created": 1234567890,
  "model": "llama-2-7b",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Hello! How can I help you today?"
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 10,
    "completion_tokens": 8,
    "total_tokens": 18
  }
}

Streaming

Enable streaming for real-time token generation:

curl https://abc123.predictor.sh/v1/chat/completions \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [{"role": "user", "content": "Tell me a story"}],
    "stream": true
  }'

Streaming Response

data: {"id":"chatcmpl-abc","choices":[{"delta":{"content":"Once"}}]}
data: {"id":"chatcmpl-abc","choices":[{"delta":{"content":" upon"}}]}
data: {"id":"chatcmpl-abc","choices":[{"delta":{"content":" a"}}]}
data: {"id":"chatcmpl-abc","choices":[{"delta":{"content":" time"}}]}
data: [DONE]

Examples

Python

from openai import OpenAI

client = OpenAI(
    base_url="https://abc123.predictor.sh/v1",
    api_key="pred_your_token"
)

# Non-streaming
response = client.chat.completions.create(
    model="default",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "What is Python?"}
    ],
    max_tokens=200
)
print(response.choices[0].message.content)

# Streaming
stream = client.chat.completions.create(
    model="default",
    messages=[{"role": "user", "content": "Tell me a joke"}],
    stream=True
)
for chunk in stream:
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="")

JavaScript

import OpenAI from 'openai';

const client = new OpenAI({
  baseURL: 'https://abc123.predictor.sh/v1',
  apiKey: 'pred_your_token',
});

// Non-streaming
const response = await client.chat.completions.create({
  model: 'default',
  messages: [{ role: 'user', content: 'Hello!' }],
});
console.log(response.choices[0].message.content);

// Streaming
const stream = await client.chat.completions.create({
  model: 'default',
  messages: [{ role: 'user', content: 'Tell me a story' }],
  stream: true,
});
for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta?.content || '');
}

curl

# Basic request
curl https://abc123.predictor.sh/v1/chat/completions \
  -H "Authorization: Bearer pred_your_token" \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [{"role": "user", "content": "Hello!"}],
    "max_tokens": 100
  }'

# With system prompt
curl https://abc123.predictor.sh/v1/chat/completions \
  -H "Authorization: Bearer pred_your_token" \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [
      {"role": "system", "content": "You are a pirate."},
      {"role": "user", "content": "Hello!"}
    ]
  }'