Skip to main content

Streaming

Set stream: true to receive the response as it is generated. The API answers with Content-Type: text/event-stream and sends one data: line per chunk, each containing a chat.completion.chunk object. The stream ends with data: [DONE].

Request​

curl https://api.serenityedge.ai/v1/chat/completions \
  -H "Authorization: Bearer $SERENITY_EDGE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "orion-plus",
    "stream": true,
    "stream_options": {"include_usage": true},
    "messages": [{"role": "user", "content": "Write a haiku about the Mediterranean."}]
  }'

Response​

data: {"id":"chatcmpl-8f0f1e2a","object":"chat.completion.chunk","created":1758542400,"model":"orion-plus","choices":[{"index":0,"delta":{"role":"assistant","content":"Blue"},"finish_reason":null}]}

data: {"id":"chatcmpl-8f0f1e2a","object":"chat.completion.chunk","created":1758542400,"model":"orion-plus","choices":[{"index":0,"delta":{"content":" waves"},"finish_reason":null}]}

data: {"id":"chatcmpl-8f0f1e2a","object":"chat.completion.chunk","created":1758542400,"model":"orion-plus","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}

data: {"id":"chatcmpl-8f0f1e2a","object":"chat.completion.chunk","created":1758542400,"model":"orion-plus","choices":[],"usage":{"prompt_tokens":14,"completion_tokens":17,"total_tokens":31,"prompt_tokens_details":{"cached_tokens":0}}}

data: [DONE]

Rebuild the message by concatenating delta.content across chunks. The first chunk carries delta.role; the last content chunk carries a non-null finish_reason.

Usage in the final chunk​

Send stream_options: {"include_usage": true} to receive token counts. The API then sends one extra chunk before [DONE] with an empty choices array and the usage object. That chunk is the only place where usage appears in a streamed response, so keep reading until [DONE] if you meter consumption on your side.

SDK examples​

Python​

from openai import OpenAI

client = OpenAI(base_url="https://api.serenityedge.ai/v1", api_key="YOUR_SERENITY_EDGE_API_KEY")

stream = client.chat.completions.create(
    model="orion-plus",
    messages=[{"role": "user", "content": "Write a haiku about the Mediterranean."}],
    stream=True,
    stream_options={"include_usage": True},
)

for chunk in stream:
    if chunk.choices:
        print(chunk.choices[0].delta.content or "", end="", flush=True)
    if chunk.usage:
        print(f"\n\nTokens: {chunk.usage.total_tokens}")

Node.js​

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.serenityedge.ai/v1",
  apiKey: process.env.SERENITY_EDGE_API_KEY,
});

const stream = await client.chat.completions.create({
  model: "orion-plus",
  messages: [{ role: "user", content: "Write a haiku about the Mediterranean." }],
  stream: true,
  stream_options: { include_usage: true },
});

for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta?.content ?? "");
  if (chunk.usage) console.log(`\n\nTokens: ${chunk.usage.total_tokens}`);
}

.NET​

using OpenAI.Chat;

ChatClient chat = client.GetChatClient("orion-plus");

await foreach (StreamingChatCompletionUpdate update in chat.CompleteChatStreamingAsync(
    new UserChatMessage("Write a haiku about the Mediterranean.")))
{
    foreach (ChatMessageContentPart part in update.ContentUpdate)
        Console.Write(part.Text);

    if (update.Usage is not null)
        Console.WriteLine($"\n\nTokens: {update.Usage.TotalTokenCount}");
}

Streaming tool calls​

When the model decides to call a tool, the chunks carry delta.tool_calls fragments instead of delta.content. Each fragment has an index; the first fragment for an index carries the id and the function name, later fragments carry pieces of function.arguments. Concatenate the argument fragments per index, then parse the JSON once finish_reason is tool_calls. See Tool calling.

Errors during a stream​

If the request is rejected before generation starts (invalid key, rate limit, bad parameter) the API returns a normal JSON error with the matching HTTP status, not an event stream. If an error occurs after the stream has started, the stream ends without a [DONE] marker; treat a stream that closes without [DONE] as failed and retry if appropriate.