Overview
Level 2: Build · Module 4: Streaming UX and Chat Interfaces · 25 min read · Core Track
Why this matters
To build production-grade streaming interfaces, frontend developers cannot treat streams as magic strings delivered by third-party black boxes. At the protocol layer, LLM token streams travel as UTF-8 encoded bytes split arbitrarily across TCP packets. A single multi-byte emoji or multi-byte character (like 🚀 or ä) might be sliced in half across two successive network chunks. If your client decoder naively runs new TextDecoder().decode(chunk) without stateful stream options, your UI will render broken replacement characters (``). Furthermore, provider SSE formats (like data: {"choices":[{"delta":{"content":"..."}}]}) require robust line-buffered parsing to prevent truncated JSON crashes.
Learning objectives
- Parse raw
ReadableStream<Uint8Array>responses using the Streams API. - Maintain UTF-8 multi-byte character boundary integrity using
TextDecoder({ stream: true }). - Implement a resilient SSE parser that handles split chunks and
[DONE]signals. - Experience real-time stream chunking and TTFT using the interactive playground.
- Contrast WebSockets vs Server-Sent Events for AI chat applications.
Interactive Laboratory: The Streaming Playground
Test the difference between buffered delivery and real-time SSE chunking, inspect chunk boundaries, and benchmark Time to First Token (TTFT) below:
Streaming vs Batch Latency Simulator
Compare Time to First Token (TTFT) and observe raw Server-Sent Event stream buffers.
Anatomy of an SSE Stream: The Wire Protocol
Server-Sent Events (SSE) format is a lightweight text protocol standardized in HTML5. A server sends events formatted as plain text blocks separated by double newlines (\n\n):
event: message
data: {"id":"chatcmpl-1","choices":[{"delta":{"content":"React"}}]}
event: message
data: {"id":"chatcmpl-1","choices":[{"delta":{"content":" 19"}}]}
data: [DONE]
Key Protocol Rules:
- Each field starts with a field name (
data:,event:,id:) followed by a space and value. - Two consecutive newline characters (
\n\n) signal the completion of an event block. data: [DONE]is the de-facto standard sentinel used by OpenAI and compatible APIs indicating the completion of the generation turn.
graph LR
subgraph Network["TCP Socket Chunks"]
Chunk1["Chunk 1: 'data: {\"choices\":[{\"delta\":{\"content\":\"Hello\"'"]
Chunk2["Chunk 2: '}}]}\\n\\ndata: {\"choices\":[{\"delta\":'"]
Chunk3["Chunk 3: '{\"content\":\" World\"}}]}\\n\\n'"]
end
Chunk1 --> Buffer["Line Buffer"]
Chunk2 --> Buffer
Chunk3 --> Buffer
Buffer --> Parser["SSE Parser"]
Parser -->|"Emit 'Hello'"| UI["React State: 'Hello'"]
Parser -->|"Emit ' World'"| UI["React State: 'Hello World'"]The UTF-8 Slicing Trap: Why stream: true is Mandatory
UTF-8 characters can consume between 1 and 4 bytes. For instance, the rocket emoji 🚀 consists of 4 bytes: [0xF0, 0x9F, 0x9A, 0x80].
If a TCP packet boundary splits this character such that Packet A contains [0xF0, 0x9F] and Packet B contains [0x9A, 0x80]:
- Calling
new TextDecoder().decode(chunkA)fails because[0xF0, 0x9F]is an invalid standalone UTF-8 sequence, producing ``. - Supplying
{ stream: true }instructsTextDecoderto hold incomplete byte sequences in an internal buffer until the next packet arrives!
// ❌ WRONG: Creates corrupt characters on multi-byte splits
const text = new TextDecoder().decode(chunk);
// ✅ CORRECT: Preserves multi-byte character state across packets
const decoder = new TextDecoder("utf-8");
const text = decoder.decode(chunk, { stream: true });Production Implementation: Writing a Stream Parser from First Principles
Here is the exact TypeScript consumer implementation without relying on external packages:
// lib/ai/sse-parser.ts
export interface ChatDelta {
content: string;
}
/**
* Consumes an SSE ReadableStream, handles chunk splitting,
* decodes UTF-8 bytes safely, and yields text deltas.
*/
export async function* parseChatStream(
response: Response
): AsyncGenerator<string, void, unknown> {
if (!response.body) {
throw new Error("Response body is null");
}
const reader = response.body.getReader();
const decoder = new TextDecoder("utf-8");
let buffer = "";
try {
while (true) {
const { done, value } = await reader.read();
if (done) break;
// 1. Decode bytes into string, preserving multi-byte boundaries
buffer += decoder.decode(value, { stream: true });
// 2. Split buffer by newlines to extract full SSE lines
const lines = buffer.split("\n");
// The last element is either empty (if ended with \n) or an incomplete line
buffer = lines.pop() || "";
for (const line of lines) {
const trimmed = line.trim();
if (!trimmed || trimmed.startsWith(":")) {
// Empty line or SSE comment (heartbeat)
continue;
}
if (trimmed.startsWith("data:")) {
const dataContent = trimmed.slice(5).trim();
// 3. Check for completion sentinel
if (dataContent === "[DONE]") {
return;
}
try {
const parsed = JSON.parse(dataContent);
const delta = parsed.choices?.[0]?.delta?.content;
if (delta) {
yield delta;
}
} catch (jsonErr) {
// Line was not complete JSON; re-append to buffer
buffer = line + "\n" + buffer;
}
}
}
}
// Flush any remaining decoder buffer
const finalFlush = decoder.decode();
if (finalFlush) {
buffer += finalFlush;
}
} finally {
reader.releaseLock();
}
}WebSockets vs Server-Sent Events (SSE)
A classic interview question asks why nearly all AI providers use SSE rather than WebSockets.
| Feature | Server-Sent Events (SSE) | WebSockets (WS) |
|---|---|---|
| Protocol | Standard HTTP/1.1 or HTTP/2 | Custom ws:// or wss:// handshake |
| Direction | Unidirectional (Server to Client) | Full Duplex (Bidirectional) |
| Reconnection | Built-in auto reconnection & event IDs | Manual heartbeat & reconnection logic |
| Proxy / Firewall | Works out of the box through enterprise proxies | Frequently blocked by corporate firewalls |
| HTTP/2 Multiplexing | Native (shares existing TCP connection) | Does not multiplex over standard HTTP/2 |
| Verdict for AI | Optimal for LLM token streaming | Overkill unless streaming voice/audio in real-time |
Interview Angle: How to Answer in System Design
- Question: "How does a client handle packet fragmentation and multi-byte UTF-8 boundaries when reading a ReadableStream?"
- Target Answer:
"A raw
ReadableStreamyields arbitraryUint8Arraybyte chunks that do not correspond 1-to-1 with JSON objects, lines, or even UTF-8 characters.First, I instantiate a single
TextDecoder('utf-8')and invoke it with{ stream: true }on every incoming chunk. This instructs the decoder to retain incomplete multi-byte code units (such as 4-byte emojis) across packet boundaries.Second, I maintain a line accumulator buffer. Because a single SSE line can arrive split across two network packets, I split the incoming buffer by
\n, pop the unfinalized fragment to retain for the next read, and parse only completedata: {...}lines. Upon encounteringdata: [DONE], I break and release the stream lock."
Quick Check
- Why must
TextDecoder.decode()be called with{ stream: true }when consuming chunked network streams? - What delimiter in the SSE wire format marks the boundary of an event block?
- What is the standard sentinel string returned by OpenAI when an SSE completion finishes?