Chat completion with a model catalog, per-model pricing, streaming, and structured outputs.
Send messages (role + content) and get a completion. You choose a model from the catalog (or use the default). The service returns message, model, token_usage (prompt/completion/total tokens), finish_reason, and an optional parsed object when json_schema is requested. Pricing in the model catalog is what you actually pay — gateway already applies its markup on the OpenAI raw rate, so list/comparison endpoints expose the post-markup rates directly. Billing per request follows the model's input/output rates from the catalog. Pass stream:true for OpenAI-compatible SSE streaming. Pass json_schema for OpenAI Structured Outputs — the response then includes a parsed field with the validated object (sync), or a final SSE chunk carrying parsed before the [DONE] marker.
Each model in core-llm-models exposes input_per_million / output_per_million / estimated_per_message. These are the customer-facing rates — the OpenAI raw cost is multiplied by the gateway markup before exposure. There is no separate markup field; the listed price is the price.
Models grouped by tier (budget, standard, premium). Each entry has model_id, display_name, description, pricing, and limits (max_tokens, context_window). default_model and aliases let you refer to models by short name.
Send messages array. Optional model, temperature, max_tokens, top_p, frequency_penalty, presence_penalty. Response: message (assistant content), model, finish_reason, token_usage.
Pass stream:true to receive an OpenAI-compatible Server-Sent Events stream of chat.completion.chunk events. A final usage chunk is sent before [DONE].
Pass json_schema {name, schema, strict} for OpenAI Structured Outputs. Sync response gains a data.parsed field (parsed JSON object). Streaming emits an extra chunk with parsed before the usage chunk.
GET core-llm-models returns models, default_model, aliases, tiers. GET core-llm-models-comparison returns a side-by-side table (model, model_id, tier, input_price, output_price, cost_per_message, is_default) plus a top-level note describing the assumed token mix.
Send conversation history as messages; get the next reply with content and token_usage. Pick a budget or premium model from the catalog.
Use stream:true to render the assistant's reply token-by-token; track final usage from the last chunk before [DONE].
Pass json_schema with strict:true to force the model to emit JSON conforming to your schema. Use data.parsed (sync) or the parsed SSE chunk (stream) instead of re-parsing message yourself.
Use core-llm-models or core-llm-models-comparison to pick by tier and estimated_per_message; aliases (e.g. default, gpt-4.1-mini, gpt4) simplify requests.
Input
messages (role, content), optional model (id or alias), temperature, max_tokens, stream, json_schema, response_format
Output
message, model, finish_reason, token_usage; optional parsed when json_schema is set. Stream variant: SSE chunks (chat.completion.chunk) ending with usage chunk and [DONE].
Prerequisites
Get the model catalog. The pricing values shown are what you actually pay (post-markup).
GET /v1/proxy/core-llm-models
Response
{
"status": "success",
"data": {
"models": [
{
"model_id": "gpt-5-mini-2025-08-07",
"display_name": "GPT-4.1 Mini",
"description": "Best balance of cost and capability. Recommended for most use cases.",
"tier": "standard",
"is_default": true,
"is_available": true,
"pricing": {
"input_per_million": 0.8,
"output_per_million": 3.2,
"input_per_1k": 0.0008,
"output_per_1k": 0.0032,
"blended_per_1k": 0.00176,
"estimated_per_message": 0.0064
},
"limits": { "max_tokens": 16384, "context_window": 128000 }
}
],
"default_model": "gpt-5-mini-2025-08-07",
"aliases": {
"default": "gpt-5-mini-2025-08-07",
"gpt-4.1-mini": "gpt-5-mini-2025-08-07",
"gpt-4.1-nano": "gpt-4.1-nano-2025-04-14",
"gpt-4.1": "gpt-4.1-2025-04-14",
"gpt4": "gpt-4o",
"gpt4-mini": "gpt-4o-mini"
},
"tiers": {
"budget": ["gpt-4.1-nano-2025-04-14", "gpt-4o-mini", "gpt-3.5-turbo"],
"standard": ["gpt-5-mini-2025-08-07"],
"premium": ["gpt-4.1-2025-04-14", "gpt-4o"]
}
}
}Use model_id or an alias (e.g. default, gpt-4.1-mini) in the completion request. Pricing values here are the rates billed to your account.
POST messages, optional model. Response includes message, model, finish_reason, token_usage. The cost object reflects the actual amount deducted from your balance.
{
"model": "default",
"messages": [
{ "role": "system", "content": "You are a helpful assistant." },
{ "role": "user", "content": "What is the capital of France?" }
],
"temperature": 0.7,
"max_tokens": 256
}Response
{
"status": "success",
"data": {
"message": "The capital of France is Paris.",
"model": "gpt-5-mini-2025-08-07",
"finish_reason": "stop",
"token_usage": { "prompt_tokens": 28, "completion_tokens": 9, "total_tokens": 37 }
},
"gateway": {
"request_id": "req_abc123",
"service": "core-llm-completions"
},
"cost": {
"units": 37.0,
"unit_price": 0.0000011,
"tokens": 0.0000406,
"balance": 4579.78
}
}- units = total tokens consumed (prompt + completion). - tokens = the amount actually deducted from your balance for this call. - unit_price = tokens / units → the blended per-token rate for this specific call (varies by input/output ratio and model). This is informational; do not use it to predict future calls. - balance = your remaining balance after this deduction.
Pass stream:true. The response is text/event-stream with OpenAI-compatible chat.completion.chunk events. A final usage chunk is sent before the [DONE] marker.
{
"model": "default",
"messages": [{ "role": "user", "content": "Write a haiku about the sea." }],
"stream": true,
"max_tokens": 256
}Response
data: {"id":"chatcmpl-abc","object":"chat.completion.chunk","model":"gpt-5-mini-2025-08-07","choices":[{"index":0,"delta":{"role":"assistant"},"finish_reason":null}]}
data: {"id":"chatcmpl-abc","object":"chat.completion.chunk","model":"gpt-5-mini-2025-08-07","choices":[{"index":0,"delta":{"content":"Salt"},"finish_reason":null}]}
data: {"id":"chatcmpl-abc","object":"chat.completion.chunk","model":"gpt-5-mini-2025-08-07","choices":[{"index":0,"delta":{"content":" wind"},"finish_reason":null}]}
data: {"id":"chatcmpl-abc","object":"chat.completion.chunk","model":"gpt-5-mini-2025-08-07","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}
data: {"id":"chatcmpl-abc","object":"chat.completion.chunk","model":"gpt-5-mini-2025-08-07","choices":[],"usage":{"prompt_tokens":12,"completion_tokens":18,"total_tokens":30}}
data: [DONE]- Each event is one line prefixed with `data: ` and terminated by a blank line. - Concatenate `choices[0].delta.content` from every chunk to reconstruct the full message. - The penultimate chunk has empty `choices` and carries the final `usage` object (OpenAI convention). - `data: [DONE]` is the stream terminator. After it, no more chunks are sent. - Billing happens once after the stream ends, using the same per-model rates as sync.
Pass json_schema with name, schema, strict. The model is forced to return JSON conforming to your schema (OpenAI Structured Outputs). The gateway parses it for you and exposes data.parsed.
{
"model": "default",
"messages": [
{ "role": "user", "content": "Extract the title and a 1-sentence summary from: 'OpenAI launches Sora 2 today, focusing on cinematic video.'" }
],
"json_schema": {
"name": "extraction_result",
"schema": {
"type": "object",
"properties": {
"title": { "type": "string" },
"summary": { "type": "string" }
},
"required": ["title", "summary"],
"additionalProperties": false
},
"strict": true
}
}Response
{
"status": "success",
"data": {
"message": "{\"title\":\"OpenAI launches Sora 2\",\"summary\":\"OpenAI released Sora 2 with a focus on cinematic video generation.\"}",
"model": "gpt-5-mini-2025-08-07",
"finish_reason": "stop",
"token_usage": { "prompt_tokens": 56, "completion_tokens": 24, "total_tokens": 80 },
"parsed": {
"title": "OpenAI launches Sora 2",
"summary": "OpenAI released Sora 2 with a focus on cinematic video generation."
}
},
"cost": { "units": 80.0, "unit_price": 0.0000010, "tokens": 0.0000832, "balance": 4579.78 }
}- `json_schema.strict: true` requires `additionalProperties: false` on every object level (OpenAI requirement). - When both `json_schema` and `response_format` are sent, `json_schema` wins. - If the model fails to produce parseable JSON, `data.parsed` is `null` (rare with strict mode). - In streaming mode, after the content chunks the gateway emits one extra chunk with empty `choices` and a top-level `parsed` object, then the usage chunk, then `data: [DONE]`.
Chat completion with messages array. Optional model (id or alias), temperature, max_tokens, top_p, frequency_penalty, presence_penalty, stream, json_schema, response_format. Returns message (assistant content), model (resolved id), finish_reason, token_usage. Optional parsed when json_schema is set. With stream:true the response is OpenAI-compatible SSE.
/v1/proxy/core-llm-completions
Get the model catalog: models (model_id, display_name, description, tier, is_default, is_available, pricing, limits), default_model, aliases, tiers (budget, standard, premium). Pricing values are post-markup — the rates the customer pays.
/v1/proxy/core-llm-models
Side-by-side comparison: model, model_id, tier, input_price, output_price (per 1M, post-markup), cost_per_message (~2K input + 1.5K output), is_default. Plus a top-level note describing the assumed token mix.
/v1/proxy/core-llm-models-comparison
Per-model pricing — see core-llm-models for each model's input_per_million / output_per_million. Listed values are the rates billed to your account (markup is already applied at exposure time). List and comparison endpoints are free.
| Service | Unit | Price |
|---|---|---|
| Chat Completion | token | Per model — see core-llm-models[].pricing |
| List Models / Comparison | item | Free |
A: data.message (assistant text), data.model (resolved id), data.finish_reason, data.token_usage. Optional data.parsed when json_schema is sent. cost contains units / unit_price / tokens / balance.
A: Yes. Aliases: default, gpt-4.1-mini, gpt-4.1-nano, gpt-4.1, gpt4, gpt4-mini. Or use the full model_id (e.g. gpt-5-mini-2025-08-07).
A: Per-model. The gateway uses the model's input/output rates as exposed in core-llm-models (those rates already include the gateway markup). The deducted amount is shown as cost.tokens in the response.
A: Set stream:true. Response is text/event-stream with OpenAI-compatible chat.completion.chunk events. Final usage chunk arrives before 'data: [DONE]'. Billing is identical and happens once when the stream ends.
A: Pass json_schema:{ name, schema, strict:true }. The gateway will add data.parsed (sync) or emit a parsed SSE chunk (stream). Use that instead of re-parsing data.message.
A: Per model. Check core-llm-models for each model's limits.context_window. max_tokens caps completion length and is capped automatically to the model's max_tokens limit.
1.2 (2026-04-27)
1.1 (2026-01-26)
1.0 (2026-01-26)