Overview
Level 2: Build · Module 4: Streaming UX and Chat Interfaces · 25 min read
Why this matters
Rendering static markdown in React is trivial with libraries like react-markdown. However, rendering markdown mid-stream is notoriously tricky. When an LLM outputs markdown tokens incrementally, the syntax tree is incomplete on nearly every render: a fenced code block might open with ```typescript but remain unclosed for 500 tokens; an unclosed table row | Col 1 | Col 2 can cause catastrophic layout reflows; an unclosed bold tag **impor causes subsequent text to flash between weights. Naive markdown re-parsing on every single token chunk (at 40 tokens per second) burns CPU cycles and causes visible UI jitter. Mastering stream-safe markdown rendering is essential for high-end AI engineering.
Learning objectives
- Diagnose mid-stream markdown rendering failure modes (unclosed blocks, table layout shifts).
- Implement memoized markdown token parsing with
remark-gfmand syntax highlighting. - Optimize render performance by batching and throttling token stream updates with
requestAnimationFrame. - Handle incomplete code blocks gracefully with streaming cursor indicators.
- Prevent syntax highlighting flashes while code is being actively generated.
Failure Modes of Streaming Markdown
Token Chunk Sequence:
Turn 1: "Here is the code:\n```typescript" -> Unclosed code block! Might eat whole page
Turn 2: "\nexport function calculate(" -> Syntax highlighter attempts partial parse
Turn 3: "x: number): number {\n return x * 2;" -> Half-formed AST
Turn 4: "\n}" -> Still unclosed
Turn 5: "\n```" -> Closed! Layout snaps into formatted containergraph TD
A["Raw Token Stream: '```typescript\nconst a = 1'"] --> B["Streaming Markdown Parser"]
B --> C{"Is Code Block Closed?"}
C -->|No| D["Virtual Closure: Append temporary '```' in memory"]
C -->|Yes| E["Standard Syntax AST"]
D --> F["Stable Prism / Shiki Highlighter"]
E --> F
F --> G["Jitter-Free React Render"]Solution 1: Virtual Markdown Closure
To prevent an open code block or unclosed bold delimiter from breaking layout formatting while streaming, we can pass text through a lightweight virtual repair function prior to feeding it into the markdown compiler:
// lib/ai/markdown-stream-repair.ts
/**
* Temporarily balances open fenced code blocks so syntax highlighters
* don't break or corrupt the layout tree while streaming.
*/
export function balanceStreamingMarkdown(text: string): string {
// 1. Check for unclosed backticks (code fences)
const backtickMatches = text.match(/```/g);
const count = backtickMatches ? backtickMatches.length : 0;
// If odd count of ```, an opening code block has no closing delimiter yet
if (count % 2 !== 0) {
return text + "\n```";
}
// 2. Check for unclosed inline backticks
const inlineMatches = text.match(/(?<!`)`(?!`)/g);
if (inlineMatches && inlineMatches.length % 2 !== 0) {
return text + "`";
}
return text;
}Solution 2: Throttling Render Cycles with requestAnimationFrame
If a fast model yields 60 tokens per second, triggering React state updates on every chunk causes 60 full component re-renders per second, re-running markdown AST generation and syntax highlighting. This crushes mobile devices and causes dropped frames.
By batching incoming chunks and flushing them to React state on animation frame intervals (16ms / 60fps), rendering stays silky smooth:
// hooks/useThrottledStream.ts
import { useState, useRef, useEffect, useCallback } from "react";
export function useThrottledStream(flushIntervalMs = 50) {
const [displayedText, setDisplayedText] = useState("");
const pendingBufferRef = useRef("");
const timerRef = useRef<NodeJS.Timeout | null>(null);
const appendDelta = useCallback((delta: string) => {
pendingBufferRef.current += delta;
if (!timerRef.current) {
timerRef.current = setTimeout(() => {
setDisplayedText(pendingBufferRef.current);
timerRef.current = null;
}, flushIntervalMs);
}
}, [flushIntervalMs]);
const flushImmediate = useCallback(() => {
if (timerRef.current) {
clearTimeout(timerRef.current);
timerRef.current = null;
}
setDisplayedText(pendingBufferRef.current);
}, []);
const reset = useCallback(() => {
pendingBufferRef.current = "";
if (timerRef.current) clearTimeout(timerRef.current);
setDisplayedText("");
}, []);
return { displayedText, appendDelta, flushImmediate, reset };
}Solution 3: The Streaming Message Component
Here is a production-grade React component combining virtual closure, ReactMarkdown, GitHub Flavored Markdown (tables), and syntax highlighting:
// components/StreamingMessage.tsx
"use client";
import React, { useMemo } from "react";
import ReactMarkdown from "react-markdown";
import remarkGfm from "remark-gfm";
import { balanceStreamingMarkdown } from "@/lib/ai/markdown-stream-repair";
interface StreamingMessageProps {
content: string;
isStreaming: boolean;
}
export const StreamingMessage = React.memo(function StreamingMessage({
content,
isStreaming,
}: StreamingMessageProps) {
// 1. Balance markdown if currently streaming
const repairedContent = useMemo(() => {
return isStreaming ? balanceStreamingMarkdown(content) : content;
}, [content, isStreaming]);
return (
<div className="prose dark:prose-invert max-w-none text-sm leading-relaxed">
<ReactMarkdown
remarkPlugins={[remarkGfm]}
components={{
table: ({ node, ...props }) => (
<div className="overflow-x-auto my-4 border border-gray-200 dark:border-gray-800 rounded-lg">
<table className="min-w-full divide-y divide-gray-200 dark:divide-gray-800" {...props} />
</div>
),
th: ({ node, ...props }) => (
<th className="px-3 py-2 bg-gray-50 dark:bg-gray-900 text-left text-xs font-semibold" {...props} />
),
td: ({ node, ...props }) => (
<td className="px-3 py-2 text-xs border-t border-gray-150 dark:border-gray-800" {...props} />
),
code: ({ node, className, children, ...props }: any) => {
const match = /language-(\w+)/.exec(className || "");
const isBlock = Boolean(match);
if (isBlock) {
return (
<div className="relative my-3 rounded-lg overflow-hidden bg-gray-950 text-gray-100 p-4 font-mono text-xs">
<div className="flex justify-between items-center text-xs text-gray-400 mb-2 border-b border-gray-800 pb-1">
<span>{match ? match[1] : "code"}</span>
{isStreaming && <span className="inline-block w-2 h-2 rounded-full bg-emerald-500 animate-pulse" />}
</div>
<pre className="overflow-x-auto">
<code>{children}</code>
</pre>
</div>
);
}
return (
<code className="px-1.5 py-0.5 rounded bg-gray-100 dark:bg-gray-800 text-pink-600 dark:text-pink-400 font-mono text-xs" {...props}>
{children}
</code>
);
},
}}
>
{repairedContent}
</ReactMarkdown>
{/* Streaming blinking cursor */}
{isStreaming && (
<span className="inline-block w-1.5 h-4 ml-1 align-middle bg-blue-600 dark:bg-blue-400 animate-pulse" />
)}
</div>
);
});Interview Angle: How to Answer in System Design
- Question: "How do you prevent UI layout shifts and performance degradation when streaming markdown and code blocks in React?"
- Target Answer:
"Streaming markdown introduces two major risks: layout instability from incomplete markdown syntax (like unclosed code fences or partial tables), and CPU thrashing from parsing the AST on every 15ms token chunk.
I solve layout stability by applying a 'virtual repair' transform that inspects unclosed backtick pairs (
```) and appends transient closing delimiters prior to AST compilation.For rendering performance, I decouple incoming network chunks from React state using a throttled buffer that flushes at 50ms intervals or via
requestAnimationFrame. Finally, I memoize message components so completed historical messages in the chat transcript do not re-render as new tokens arrive."
Quick Check
- Why does an unclosed
```code fence cause layout issues in streaming markdown? - What is the advantage of throttling state updates to 50ms rather than updating state on every single token chunk?
- How does virtual markdown repair work without corrupting the raw message history?