Chat completions

POST /v1/chat/completions

Conversational text generation. Supports streaming, function calling, vision, structured output, and prompt caching. Wire-format-compatible with the standard chat-completions schema.

When to use which model

Model Use when
epithre-omni Default. General chat with vision, agentic tool-use, extended thinking.
epithre-prme Long context (180K). Full-codebase or long-document analysis.
epithre-lyt Fast, cheap, high-throughput. Multimodal: image, audio, video inputs.

Request body

Field Type Required Description
model string yes epithre-omni, epithre-prme, or epithre-lyt.
messages array yes Standard chat messages. Roles: system, user, assistant, tool. Each message has content (string or content-block array).
max_tokens int no Max output tokens. Default model-dependent. Max 16384 for omni/prme, 4096 for lyt.
temperature float no 0.0-2.0, default 1.0. Lower means more deterministic.
top_p float no 0.0-1.0 nucleus sampling, default 1.0.
stream bool no If true, returns SSE chunks. Final chunk includes usage when stream_options.include_usage=true.
stream_options object no {"include_usage": true} to get token counts in the final SSE chunk.
tools array no Function definitions. Standard tools schema (name + description + JSON-Schema parameters).
tool_choice string or object no "auto" (default), "none", "required", or {"type":"function","function":{"name":"..."}}.
response_format object no {"type":"json_object"} for any valid JSON; {"type":"json_schema", "json_schema":{...}} for strict schema-conformant JSON. See structured output guide.
chat_template_kwargs object no e.g. {"enable_thinking": true} for extended thinking on omni/prme. Default false.
seed int no For reproducibility (best-effort).
stop string or array no Stop sequences (up to 4).
frequency_penalty float no -2 to 2, default 0.
presence_penalty float no -2 to 2, default 0.

Message content shapes

A message's content can be a plain string or an array of content blocks.

String form (most common):

{"role": "user", "content": "Halo, apa kabar?"}

Content-block array (required for vision, tool messages with images, or cache_control):

{"role": "user", "content": [
  {"type": "text", "text": "Apa yang ada di gambar ini?"},
  {"type": "image_url", "image_url": {"url": "data:image/jpeg;base64,..."}}
]}

With cache_control marker (for prompt caching):

{"role": "system", "content": [
  {"type": "text",
   "text": "<long stable system prompt>",
   "cache_control": {"type": "ephemeral"}}
]}

The marker MUST sit on a content block inside a list-form content. Plain-string content with a sibling cache_control field is silently ignored. Top-level tools aren't covered by markers — see prompt caching guide for placement rules and the agentic-loop pattern.

Response shape (non-streaming)

{
  "id": "chatcmpl-xxxxxx",
  "object": "chat.completion",
  "created": 1778455870,
  "model": "epithre-omni",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Ibu kota Jepang adalah Tokyo."
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 18,
    "completion_tokens": 9,
    "total_tokens": 27,
    "cache_creation_input_tokens": 0,
    "cache_read_input_tokens": 0
  }
}

finish_reason values: "stop" (natural end or stop sequence), "length" (hit max_tokens), "tool_calls" (model wants to call a tool), "content_filter" (rare; safety filter).

usage.cache_creation_input_tokens and usage.cache_read_input_tokens are always present.

Streaming response (SSE)

data: {"id":"chatcmpl-x","choices":[{"index":0,"delta":{"role":"assistant","content":""}}]}

data: {"id":"chatcmpl-x","choices":[{"index":0,"delta":{"content":"Ibu kota "}}]}

data: {"id":"chatcmpl-x","choices":[{"index":0,"delta":{"content":"Jepang adalah "}}]}

data: {"id":"chatcmpl-x","choices":[{"index":0,"delta":{"content":"Tokyo."}}]}

data: {"id":"chatcmpl-x","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}

data: {"id":"chatcmpl-x","choices":[],"usage":{"prompt_tokens":18,"completion_tokens":9,"total_tokens":27,"cache_creation_input_tokens":0,"cache_read_input_tokens":0}}

data: [DONE]

See streaming guide.

Examples

Basic chat

resp = client.chat.completions.create(
    model="epithre-omni",
    messages=[{"role": "user", "content": "Apa ibu kota Jepang?"}],
)
print(resp.choices[0].message.content)

Streaming

stream = client.chat.completions.create(
    model="epithre-omni",
    messages=[{"role": "user", "content": "Ceritakan sejarah singkat Jakarta."}],
    stream=True,
)
for chunk in stream:
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)

Tool use

tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get current weather.",
        "parameters": {
            "type": "object",
            "properties": {"city": {"type": "string"}},
            "required": ["city"],
        },
    },
}]

resp = client.chat.completions.create(
    model="epithre-omni",
    messages=[{"role": "user", "content": "Cuaca Jakarta hari ini?"}],
    tools=tools,
)
print(resp.choices[0].message.tool_calls)

Vision

import base64
img_b64 = base64.b64encode(open("invoice.jpg", "rb").read()).decode()

resp = client.chat.completions.create(
    model="epithre-omni",
    messages=[{"role": "user", "content": [
        {"type": "text", "text": "Ekstrak total dari invoice ini."},
        {"type": "image_url", "image_url": {"url": f"data:image/jpeg;base64,{img_b64}"}},
    ]}],
)

Structured output

resp = client.chat.completions.create(
    model="epithre-omni",
    messages=[{"role": "user", "content": "Ekstrak: 'Dimas, umur 29, dari Jakarta'"}],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "person",
            "strict": True,
            "schema": {
                "type": "object",
                "properties": {
                    "name": {"type": "string"},
                    "age":  {"type": "integer"},
                    "city": {"type": "string"},
                },
                "required": ["name", "age", "city"],
                "additionalProperties": False,
            },
        },
    },
)

Errors

HTTP Code Cause
400 model_not_found Bad model value. Use one of epithre-omni/epithre-prme/epithre-lyt.
400 invalid_request_error Empty messages, malformed body, invalid tool schema.
429 backend_busy Aggregate traffic at backend cap. Retry in ~1 second.
429 concurrency_exceeded Your key's concurrent cap hit. Reduce parallelism.
500 backend_error Inference failure. Rare. Retry once.
504 backend_timeout Inference too slow. Try shorter prompt.

Full catalogue: errors.

See also