Stream chat completions

A non-streaming chat completion returns nothing until the whole answer is generated. For anything a person watches — a chat UI, an agent trace, a long summary — that wait is the product. Streaming sends the answer as it is produced, so the first token reaches your user in a fraction of the total time.

This page shows the streaming call in three languages, how to read the incremental delta objects, and what to do when the connection drops mid-stream.

Prerequisites

• An account with a funded balance. See Get 0G and fund your account.
• An sk- API key from the 0G Private Computer Console, exported as ZG_API_KEY. Keys are shown once; rotate one immediately if it may have leaked.

The examples use glm-5, an OpenAI-format chat model.

Stream the response

Set stream: true and iterate over the chunks. The wire format is standard OpenAI server-sent events — nothing here is Router-specific.

cURL

curl https://router-api.0g.ai/v1/chat/completions \
  --max-time 120 \
  --fail-with-body \
  --no-buffer \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $ZG_API_KEY" \
  -d '{
    "model": "glm-5",
    "messages": [{"role": "user", "content": "Explain trusted execution environments in three sentences."}],
    "stream": true
  }'

--no-buffer is what makes the output appear incrementally; without it curl buffers and you see the whole stream at once. --fail-with-body makes curl exit non-zero on an HTTP error while still printing the error JSON.

TypeScript

import OpenAI from "openai";

const apiKey = process.env.ZG_API_KEY;
if (!apiKey) throw new Error("ZG_API_KEY is not set");

const client = new OpenAI({
  apiKey,
  baseURL: "https://router-api.0g.ai/v1",
  timeout: 120_000,
  maxRetries: 0, // classify failures below instead of retrying blindly
});

async function streamAnswer(prompt: string): Promise<string> {
  let text = "";

  try {
    const stream = await client.chat.completions.create({
      model: "glm-5",
      messages: [{ role: "user", content: prompt }],
      stream: true,
    });

    for await (const chunk of stream) {
      const delta = chunk.choices[0]?.delta;
      if (delta?.content) {
        text += delta.content;
        process.stdout.write(delta.content);
      }
    }
  } catch (err) {
    if (err instanceof OpenAI.APIError) {
      // The request never started streaming: status and code are meaningful.
      throw new Error(`router error ${err.status} (${(err.error as { code?: string } | undefined)?.code})`);
    }
    // The connection broke mid-stream. `text` holds the partial answer.
    throw new StreamInterrupted(text, err);
  }

  return text;
}

class StreamInterrupted extends Error {
  constructor(readonly partial: string, readonly cause: unknown) {
    super(`stream interrupted after ${partial.length} characters`);
  }
}

await streamAnswer("Explain trusted execution environments in three sentences.");

Python

import os

from openai import APIError, OpenAI

api_key = os.environ["ZG_API_KEY"]

client = OpenAI(
    api_key=api_key,
    base_url="https://router-api.0g.ai/v1",
    timeout=120.0,
    max_retries=0,  # classify failures below instead of retrying blindly
)


class StreamInterrupted(Exception):
    def __init__(self, partial: str, cause: Exception) -> None:
        super().__init__(f"stream interrupted after {len(partial)} characters")
        self.partial = partial
        self.cause = cause


def stream_answer(prompt: str) -> str:
    text = ""
    try:
        stream = client.chat.completions.create(
            model="glm-5",
            messages=[{"role": "user", "content": prompt}],
            stream=True,
        )
        for chunk in stream:
            delta = chunk.choices[0].delta
            if delta.content:
                text += delta.content
                print(delta.content, end="", flush=True)
    except APIError as err:
        # The request never started streaming: status and code are meaningful.
        raise RuntimeError(f"router error: {err}") from err
    except Exception as err:
        # The connection broke mid-stream. `text` holds the partial answer.
        raise StreamInterrupted(text, err) from err

    return text


stream_answer("Explain trusted execution environments in three sentences.")

To go back to a single blocking response, drop stream (or set it to false) and read choices[0].message.content instead of iterating.

Expected output

Each server-sent event carries one chunk. Consecutive delta.content values concatenate into the final answer; the last chunk sets finish_reason and carries an empty delta.

{
  "id": "chatcmpl-2a23c437-775d-4f28-b25c-824e8ba14e0b",
  "object": "chat.completion.chunk",
  "model": "glm-5",
  "created": 1784535735,
  "choices": [
    {
      "index": 0,
      "delta": { "role": "assistant", "content": null, "reasoning_content": "1" },
      "finish_reason": null,
      "logprobs": null
    }
  ],
  "usage": null
}

The chunk that ends the text sets finish_reason and an empty content:

{
  "choices": [
    { "index": 0, "delta": { "content": "", "reasoning_content": null }, "finish_reason": "stop", "logprobs": null }
  ],
  "usage": null
}

One more chunk follows it, with no choices at all, carrying the totals and the trace block:

{
  "choices": [],
  "usage": {
    "prompt_tokens": 10,
    "completion_tokens": 922,
    "completion_tokens_details": { "reasoning_tokens": 919 },
    "total_tokens": 932
  },
  "x_0g_trace": {
    "request_id": "8312e180-3ecb-43d3-8fd9-06fe5777a59a",
    "provider": "0xB01EBd79c3fd63ff52fD47C3935119601EEe2FdB",
    "billing": { "input_cost": "29900000000000", "output_cost": "12428560000000000", "total_cost": "12458460000000000" }
  }
}

Live responses, 2026-07-20. Then the raw SSE stream terminates with data: [DONE]; both SDKs consume it for you and end the iteration.

Two things this shape implies. A loop that breaks on finish_reason: "stop" never sees the usage and cost, so read through to [DONE] if you want them. And a loop that assumes choices[0] exists on every chunk throws on the final one — guard it. Reasoning models also stream reasoning_content before any content arrives, which is why a stream can look idle for several seconds and still be working.

Handle a broken stream

Two failures look alike from the outside and need opposite handling.

The request never started. The Router rejected it before any bytes of the body were produced, so you get a normal HTTP error with a status and a code — 401, 402, 429, 503. Nothing was billed for output. Apply the usual policy: honour Retry-After on 429, fix the request on 4xx, do not hammer a 503.

The stream stopped mid-answer. The connection dropped, the timeout fired, or the process was cancelled. You hold a partial answer and no finish_reason. There is no resume: a retry re-runs the whole completion from the start and is billed again.

Decide up front what a partial answer means for your application:

• Discard it and retry once if the output must be complete or structured, such as JSON or a tool call. A truncated JSON object is worse than no object.
• Keep it and mark it truncated if the output is prose being displayed to a person. Track that you never saw a finish_reason, so downstream code cannot mistake it for a finished answer.

Absence of finish_reason is the signal

A stream that ends without a finish_reason is not a complete answer, even when the text looks like one. Record the flag alongside the text rather than inferring completeness from the content.

Streaming chunks do not carry the x_0g_trace block that non-streaming responses return, so per-request billing and provider data are not available on a streamed call. Use a non-streaming request when you need them inline.

Common errors

Status	code	Cause	Fix
400	invalid_body	Malformed request, for example stream sent as a string	Fix the body; do not retry unchanged
401	invalid_api_key	ZG_API_KEY unset, wrong, revoked, or an mk- key	Inference needs a live sk- key
402	insufficient_balance	Deposit exhausted	Top up, then retry
429	rate_limit_exceeded	Too many requests	Wait for Retry-After seconds
502	provider_error	Provider failed after failover was exhausted	Short retry is reasonable
503	no_available_provider	No healthy provider for the model	Retry with backoff or switch model
—	—	Output stops mid-sentence, no finish_reason	Connection dropped; see above

Full list with retry guidance: Error codes.

Next steps

• Trust modes — constrain which providers may serve a streamed request
• Handle failover and retries — a retry policy that distinguishes transient from structural failures