OpenAI-compatible
Requests and responses match the OpenAI Chat Completions shape. Reuse your existing OpenAI client — just change the base URL and key.
CatalogLatency First token in ~0.5–2 s (streaming); full response scales with answer length Async false Rate Limit Per API key Max Input Per the chosen model's context window — see List Models
cn8/llm
Send OpenAI-style chat messages and get a completion back from the model you choose —
Learn more about LLM Completions LLM completionssyncInstant previewllmtextgeneration
Ask the model
Additional settings
Temperature0.7
Max tokens~768 words
Top-pAuto
⌘/Ctrl + Enter to run
Result
Your conversation shows up here
You ask on the left — question and answer appear as chat.
Overview
Send OpenAI-style chat messages and get a completion back from the model you choose — GPT, Claude, Gemini and others, all behind a single endpoint and a single API key. The request and response follow the OpenAI Chat Completions format, so you can point an existing OpenAI client at this endpoint and change one line. Stream the answer token by token with Server-Sent Events, or get it in one response. Browse the available models (and compare them side by side) before you pick one.
Key capabilities
Many models, one endpoint
Choose GPT, Claude, Gemini and more by setting the model field. Switch models without changing your code.
Streaming or single response
Set stream:true to receive Server-Sent chat.completion.chunk events as the model types, or omit it to get the full answer at once.
Browse and compare models
List the available models with their capabilities, or get a side-by-side comparison, before you choose.
When to use it
Chat and assistants
Power chatbots, copilots and assistants with multi-turn conversations.
Content generation
Draft, rewrite, summarize or translate text from a prompt.
Structured extraction
Ask the model to read text and return structured fields (with your own instructions in a system message).
Drop-in OpenAI replacement
Move an existing OpenAI integration over by changing the base URL and key.
Input & output
input
A model id and a list of chat messages (system / user / assistant), with optional temperature, max_tokens and stream.
JSON body (OpenAI Chat Completions format)
output
The standard CN8 response envelope: the assistant's reply in data.message plus data.token_usage. With stream:true you instead receive an SSE stream of OpenAI-style chat.completion.chunk events.
JSONSSE stream (stream:true)
Guides & tips
OpenAI-style requests, CN8-style responses
- The request body follows the OpenAI Chat Completions format: model, messages, stream, temperature, max_tokens, top_p. If you already build OpenAI-style payloads, they work as-is.
- The non-streaming response is the standard CN8 envelope, not an OpenAI chat.completion object: read the reply from
data.messageand token counts fromdata.token_usage. An unmodified OpenAI SDK won't parse it — send the request with your own HTTP client. - Streaming (
stream: true) does emit OpenAI-compatiblechat.completion.chunkSSE events, so existing OpenAI stream-parsing code works there. - Switch models any time by changing the
modelfield — no other code changes needed.
Streaming
- Set
stream: trueto receive Server-Sent Events. - Each event is a
chat.completion.chunk; appendchoices[0].delta.contentas chunks arrive. - The stream ends with
data: [DONE]. The final chunk before it carries the tokenusage.
Choosing a model
- Call List Models for the current ids and context windows, or Compare Models for a side-by-side view.
- Pick a smaller, faster model for short tasks and a larger one for long context or harder reasoning.
Specs
Schema
Request body
messagesrequiredarrayWhat you want the model to answer. In the API this is an OpenAI-style array: [{ role: 'system' | 'user' | 'assistant', content: string }].
modelrequiredstring · default: gpt-4.1-miniWhich model answers. Prices differ per model — the live list shows each one's rate.
temperaturenumber · default: 0.7Lower is focused and repeatable, higher is more creative. Reasoning models ignore this.
max_tokensinteger · default: 512Upper bound on the reply. ~1 token is 4 characters; 512 is a few paragraphs.
streamboolean · default: falseWhen true the reply arrives token by token as SSE (chat.completion.chunk events) instead of one JSON body.
json_schemaobjectForce the reply to match a JSON Schema: { name, strict, schema }. With strict=true every object needs additionalProperties=false and all properties listed in required (use anyOf-with-null for optionals). OpenAI models enforce this natively and return guaranteed raw JSON; Claude models follow the schema faithfully but may wrap the JSON in a markdown fence - strip it before parsing.
top_pnumberAlternative to temperature — limits the token pool to the most likely mass. Change one or the other, not both.
Pricing
Chat completions are billed per token at the selected model's rate (reported by the upstream per call — cheap models bill far below the nominal ceiling). Listing and comparing models is free.
| Service | Unit | Price |
|---|---|---|
| Chat Completion | token | per token, per model — see List Models for live rates |
| List Models | item | Free |
| Compare Models | item | Free |
Completions are billed on total_tokens (prompt + completion).
Streaming and non-streaming are billed the same — by tokens, not by request.