# Streaming

> Receive chat completions as server-sent events, with token usage in the final chunk.

Source: https://docs.serenityedge.ai/docs/streaming/

Set `stream: true` to receive the response as it is generated. The API answers with `Content-Type: text/event-stream` and sends one `data:` line per chunk, each containing a `chat.completion.chunk` object. The stream ends with `data: [DONE]`.

## Request

```bash
curl https://api.serenityedge.ai/v1/chat/completions \
  -H "Authorization: Bearer $SERENITY_EDGE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "orion-plus",
    "stream": true,
    "stream_options": {"include_usage": true},
    "messages": [{"role": "user", "content": "Write a haiku about the Mediterranean."}]
  }'
```

## Response

```text
data: {"id":"chatcmpl-8f0f1e2a","object":"chat.completion.chunk","created":1758542400,"model":"orion-plus","choices":[{"index":0,"delta":{"role":"assistant","content":"Blue"},"finish_reason":null}]}

data: {"id":"chatcmpl-8f0f1e2a","object":"chat.completion.chunk","created":1758542400,"model":"orion-plus","choices":[{"index":0,"delta":{"content":" waves"},"finish_reason":null}]}

data: {"id":"chatcmpl-8f0f1e2a","object":"chat.completion.chunk","created":1758542400,"model":"orion-plus","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}

data: {"id":"chatcmpl-8f0f1e2a","object":"chat.completion.chunk","created":1758542400,"model":"orion-plus","choices":[],"usage":{"prompt_tokens":14,"completion_tokens":17,"total_tokens":31,"prompt_tokens_details":{"cached_tokens":0}}}

data: [DONE]
```

Rebuild the message by concatenating `delta.content` across chunks. The first chunk carries `delta.role`; the last content chunk carries a non-null `finish_reason`.

## Usage in the final chunk

Send `stream_options: {"include_usage": true}` to receive token counts. The API then sends one extra chunk before `[DONE]` with an empty `choices` array and the `usage` object. That chunk is the only place where usage appears in a streamed response, so keep reading until `[DONE]` if you meter consumption on your side.

## SDK examples

### Python

```python
from openai import OpenAI

client = OpenAI(base_url="https://api.serenityedge.ai/v1", api_key="YOUR_SERENITY_EDGE_API_KEY")

stream = client.chat.completions.create(
    model="orion-plus",
    messages=[{"role": "user", "content": "Write a haiku about the Mediterranean."}],
    stream=True,
    stream_options={"include_usage": True},
)

for chunk in stream:
    if chunk.choices:
        print(chunk.choices[0].delta.content or "", end="", flush=True)
    if chunk.usage:
        print(f"\n\nTokens: {chunk.usage.total_tokens}")
```

### Node.js

```javascript
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.serenityedge.ai/v1",
  apiKey: process.env.SERENITY_EDGE_API_KEY,
});

const stream = await client.chat.completions.create({
  model: "orion-plus",
  messages: [{ role: "user", content: "Write a haiku about the Mediterranean." }],
  stream: true,
  stream_options: { include_usage: true },
});

for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta?.content ?? "");
  if (chunk.usage) console.log(`\n\nTokens: ${chunk.usage.total_tokens}`);
}
```

### .NET

```csharp
using OpenAI.Chat;

ChatClient chat = client.GetChatClient("orion-plus");

await foreach (StreamingChatCompletionUpdate update in chat.CompleteChatStreamingAsync(
    new UserChatMessage("Write a haiku about the Mediterranean.")))
{
    foreach (ChatMessageContentPart part in update.ContentUpdate)
        Console.Write(part.Text);

    if (update.Usage is not null)
        Console.WriteLine($"\n\nTokens: {update.Usage.TotalTokenCount}");
}
```

## Streaming tool calls

When the model decides to call a tool, the chunks carry `delta.tool_calls` fragments instead of `delta.content`. Each fragment has an `index`; the first fragment for an index carries the `id` and the function `name`, later fragments carry pieces of `function.arguments`. Concatenate the argument fragments per index, then parse the JSON once `finish_reason` is `tool_calls`. See [Tool calling](https://docs.serenityedge.ai/docs/tool-calling/).

## Errors during a stream

If the request is rejected before generation starts (invalid key, rate limit, bad parameter) the API returns a normal JSON error with the matching HTTP status, not an event stream. If an error occurs after the stream has started, the stream ends without a `[DONE]` marker; treat a stream that closes without `[DONE]` as failed and retry if appropriate.