Structured output

response_format constrains the model to emit valid JSON, optionally schema-conformant. Eliminates parse-retry loops in your code.

Two modes:

{"type": "json_object"} - any valid JSON.
{"type": "json_schema", "json_schema": {...}} - strict, schema-validated.

Both work on epithre-omni and epithre-prme. Our grammar enforcement layer is fast and reliable across the typical prompt-length range; for most workloads it just works.

JSON mode (any valid JSON)

resp = client.chat.completions.create(
    model="epithre-omni",
    messages=[
        {"role": "system", "content": "Reply with valid JSON only."},
        {"role": "user", "content": "Ekstrak: 'Dimas, umur 29, dari Jakarta'"},
    ],
    response_format={"type": "json_object"},
)
import json
data = json.loads(resp.choices[0].message.content)
# {"name": "Dimas", "age": 29, "city": "Jakarta"}

The model picks the keys. Useful when shape is loosely defined or you want exploration. For production, use json_schema.

Strict JSON schema

Provide a JSON Schema; output is guaranteed to validate.

resp = client.chat.completions.create(
    model="epithre-omni",
    messages=[
        {"role": "user", "content": "Ekstrak: 'Sari, 34 tahun, dokter, Surabaya'"},
    ],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "person_extract",
            "strict": True,
            "schema": {
                "type": "object",
                "properties": {
                    "name":       {"type": "string"},
                    "age":        {"type": "integer", "minimum": 0, "maximum": 150},
                    "profession": {"type": "string"},
                    "city":       {"type": "string"}
                },
                "required": ["name", "age", "profession", "city"],
                "additionalProperties": False,
            },
        },
    },
)
import json
data = json.loads(resp.choices[0].message.content)
# {"name": "Sari", "age": 34, "profession": "dokter", "city": "Surabaya"}

Schema features supported:

type: string, integer, number, boolean, array, object, null
enum: restricted vocab
required, additionalProperties
Nested object and array (with items)
minimum/maximum for numbers, minLength/maxLength for strings (best-effort)
pattern for strings (best-effort - the grammar tries but isn't fully regex-compliant)

Enum classification example

Particularly clean pattern for sentiment / category labels:

resp = client.chat.completions.create(
    model="epithre-lyt",
    messages=[
        {"role": "system", "content": "Classify sentiment of Indonesian product reviews."},
        {"role": "user", "content": "Barang nyampe tapi salah warna. Komplain blm direspon."},
    ],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "sentiment",
            "strict": True,
            "schema": {
                "type": "object",
                "properties": {
                    "sentiment": {"type": "string", "enum": ["positif", "netral", "negatif"]},
                    "confidence": {"type": "number", "minimum": 0, "maximum": 1},
                },
                "required": ["sentiment", "confidence"],
                "additionalProperties": False,
            },
        },
    },
)
# {"sentiment": "negatif", "confidence": 0.92}

Using epithre-lyt for this kind of high-volume classification keeps cost minimal.

Nested structure

schema = {
    "type": "object",
    "properties": {
        "invoice_number": {"type": "string"},
        "vendor": {
            "type": "object",
            "properties": {
                "name":  {"type": "string"},
                "npwp":  {"type": "string", "pattern": r"^\d{2}\.\d{3}\.\d{3}\.\d-\d{3}\.\d{3}$"},
            },
            "required": ["name"],
            "additionalProperties": False,
        },
        "items": {
            "type": "array",
            "items": {
                "type": "object",
                "properties": {
                    "description": {"type": "string"},
                    "quantity":    {"type": "integer", "minimum": 1},
                    "unit_price":  {"type": "number"},
                    "subtotal":    {"type": "number"},
                },
                "required": ["description", "quantity", "unit_price", "subtotal"],
                "additionalProperties": False,
            },
        },
        "total":  {"type": "number"},
        "ppn":    {"type": "number"},
        "grand_total": {"type": "number"},
    },
    "required": ["invoice_number", "vendor", "items", "total", "grand_total"],
    "additionalProperties": False,
}

Caveats and stability notes

Long-prompt structured output reliability (updated May 2026): the prior stall on epithre-omni with very long prompts + strict schemas is resolved. Schema-constrained generation is now stable across the typical prompt-length range. If you hit unusual latency (>30s for what should be quick), reduce prompt size or fall back to json_object as a defensive measure.
Streaming + structured output works but you receive partial-but-valid JSON during the stream. Most parsers won't handle that gracefully. If you need streaming, prefer streaming text and parse at the end.
json_schema is strict by default when strict: true. The model will fail (return null content with length finish_reason) rather than produce invalid JSON if it can't fit the schema. Always check finish_reason == "stop" before parsing.
Combination with tool calling is supported (you can pass both tools and response_format), but tools typically beat response_format. If the model decides to call a tool, the response_format constraint doesn't apply to that turn. Plan your flow accordingly.

What works well

Entity extraction from documents
Classification with confidence scores
Multi-field form filling (invoices, KTP, receipts)
Structured agent observations / decisions

What works less well

Free-form long text inside JSON fields (e.g. a 2000-word summary as a JSON string value). The grammar layer is slower per token, and you don't really need JSON for that case. Just emit text.
Schemas with deep recursion (e.g. tree of arbitrary depth). The grammar gets slow to compile.
Schemas with many oneOf / anyOf branches. Compile time grows. Flatten if possible.

Cookbook: structured extraction - more concrete patterns.
Cookbook: classification - enum classifier patterns.
Tool use guide - if tool_calls fit better than response_format for your use case.