Overview
Level 1: Foundations · Module 3: Calling LLM APIs from Next.js · 20 min read
Why this matters
Next.js App Router gives frontend engineers two primary server-side execution boundaries for running server code: Route Handlers (app/api/.../route.ts) and Server Actions ("use server"). When integrating Large Language Models, choosing between them is not a matter of personal preference; it directly impacts how streaming responses are transferred, how edge vs node runtimes are configured, how cancellation signals are handled, and how client-side state hooks integrate. Selecting the wrong boundary often causes buffered non-streaming responses, memory exhaustion on serverless lambdas, or broken request cancellation.
Learning objectives
- Compare Route Handlers vs Server Actions across latency, streaming support, and ergonomics.
- Implement a robust Route Handler returning an SSE
ReadableStream. - Implement a typed Server Action for one-shot structured transformations.
- Configure Next.js execution runtimes (
nodejsvsedge) and max duration timeouts. - Answer architectural interview questions about Next.js server execution models for AI workloads.
Comparison: Route Handlers vs Server Actions
| Capability / Trait | Route Handlers (route.ts) | Server Actions ("use server") |
|---|---|---|
| Primary Protocol | Standard HTTP (GET, POST) | RPC over HTTP POST with React Flight protocol |
| Streaming Style | Standard Web Streams (ReadableStream, SSE) | RSC Action streaming (native in React 19) |
| Client Hook Compatibility | Vercel AI SDK useChat(), fetch(), EventSource | useActionState(), form actions, direct function invocation |
| Payload Format | Custom JSON, SSE text/event-stream, binary | Serialized React Server Components or JSON-compatible primitives |
| Best Used For | Real-time chat streaming, public APIs, webhooks | Form submissions, inline classification, structured mutations |
| Cancellation Handling | Full access to request.signal (AbortSignal) | Implicit closure over invocation context |
graph TD
User["Client (Browser)"]
subgraph RouteHandlerPath["Route Handler Path (Real-time Stream)"]
User -->|"fetch('/api/chat', { signal })"| RH["app/api/chat/route.ts"]
RH -->|"ReadableStream (SSE chunks)"| LLM1["AI Provider API"]
LLM1 -.->|"Chunks 25 tok/sec"| RH
RH -.->|"Stream directly to useChat()"| User
end
subgraph ServerActionPath["Server Action Path (One-Shot Mutation)"]
User -->|"await classifyTicketAction(data)"| SA["actions/classify.ts ('use server')"]
SA -->|"Single JSON-mode Call"| LLM2["AI Provider API"]
LLM2 -->|"Typed JSON"| SA
SA -->|"Validated Object"| User
endImplementation 1: The Streaming Route Handler
When building interactive conversational agents or long-form document generators, Route Handlers provide direct control over standard Web Response headers and byte streams:
// app/api/chat/route.ts
import { NextRequest, NextResponse } from "next/server";
import { z } from "zod";
// Required when running on serverless platforms to prevent 10s default killing
export const maxDuration = 60; // 60 seconds
export const dynamic = "force-dynamic";
const ChatRequestSchema = z.object({
messages: z.array(
z.object({
role: z.enum(["system", "user", "assistant"]),
content: z.string().min(1).max(10000),
})
),
model: z.string().default("gpt-4o"),
});
export async function POST(req: NextRequest) {
try {
// 1. Validate incoming JSON payload
const body = await req.json();
const parseResult = ChatRequestSchema.safeParse(body);
if (!parseResult.success) {
return NextResponse.json(
{ error: "Invalid request payload", details: parseResult.error.issues },
{ status: 400 }
);
}
const { messages, model } = parseResult.data;
// 2. Forward to LLM with client abort signal forwarding
const response = await fetch("https://api.openai.com/v1/chat/completions", {
method: "POST",
headers: {
"Content-Type": "application/json",
Authorization: `Bearer ${process.env.OPENAI_API_KEY}`,
},
body: JSON.stringify({
model,
messages,
stream: true,
}),
// Essential: If user closes browser or clicks stop, cancel upstream upstream call!
signal: req.signal,
});
if (!response.ok) {
const errText = await response.text();
return NextResponse.json(
{ error: `Upstream AI provider error: ${response.status}`, details: errText },
{ status: response.status }
);
}
// 3. Pipe upstream SSE stream directly to client
return new Response(response.body, {
headers: {
"Content-Type": "text/event-stream; charset=utf-8",
"Cache-Control": "no-cache, no-transform",
Connection: "keep-alive",
"X-Accel-Buffering": "no", // Disables NGINX buffering in production proxies
},
});
} catch (error: any) {
if (error.name === "AbortError") {
return new Response("Client aborted", { status: 499 });
}
return NextResponse.json(
{ error: "Internal Server Error", message: error.message },
{ status: 500 }
);
}
}Implementation 2: The Structured Server Action
For discrete mutations—such as categorizing an issue, summarizing a selected paragraph, or extracting form metadata—Server Actions provide superior developer ergonomics with zero boilerplate API routes:
// app/actions/extract-ticket.ts
"use server";
import { z } from "zod";
const TicketSchema = z.object({
title: z.string().describe("Clear, concise summary of the issue"),
urgency: z.enum(["LOW", "MEDIUM", "HIGH", "CRITICAL"]),
department: z.enum(["BILLING", "TECH_SUPPORT", "SALES", "SECURITY"]),
tags: z.array(z.string()).max(5),
});
export type TicketExtraction = z.infer<typeof TicketSchema>;
export interface ActionResult<T> {
success: boolean;
data?: T;
error?: string;
}
export async function extractTicketAction(
rawInput: string
): Promise<ActionResult<TicketExtraction>> {
if (!rawInput || rawInput.trim().length < 10) {
return { success: false, error: "Input is too short to analyze." };
}
try {
const response = await fetch("https://api.openai.com/v1/chat/completions", {
method: "POST",
headers: {
"Content-Type": "application/json",
Authorization: `Bearer ${process.env.OPENAI_API_KEY}`,
},
body: JSON.stringify({
model: "gpt-4o-mini",
response_format: { type: "json_object" },
messages: [
{
role: "system",
content: `You are a triage classifier. Extract structured JSON adhering to this schema:
{
"title": string,
"urgency": "LOW" | "MEDIUM" | "HIGH" | "CRITICAL",
"department": "BILLING" | "TECH_SUPPORT" | "SALES" | "SECURITY",
"tags": string[]
}`
},
{ role: "user", content: rawInput },
],
temperature: 0.1,
}),
});
if (!response.ok) {
throw new Error(`Model API rejected with status ${response.status}`);
}
const payload = await response.json();
const parsedJson = JSON.parse(payload.choices[0].message.content);
const validated = TicketSchema.parse(parsedJson);
return { success: true, data: validated };
} catch (err: any) {
console.error("[extractTicketAction] Failure:", err);
return {
success: false,
error: err instanceof z.ZodError ? "Schema validation failed" : "Model processing failed",
};
}
}Runtimes: Edge vs Node.js in Next.js
You can choose the server execution environment per file using export const runtime = 'nodejs' | 'edge'. Here are the real tradeoffs:
┌──────────────────────────────────────┬──────────────────────────────────────┐
│ Node.js Runtime │ Edge Runtime (V8 Isolates) │
├──────────────────────────────────────┼──────────────────────────────────────┤
│ • Full Node.js stdlib (fs, crypto) │ • Sub-10ms cold starts │
│ • Official SDK compatibility │ • Globally distributed near user │
│ • 15s to 300s timeout ceilings │ • Strict memory & binary limitations │
│ • Standard serverless lambda compute │ • No dynamic code evaluation │
│ • Best for: Heavy RAG, PDF parsing │ • Best for: Fast proxying of streams │
└──────────────────────────────────────┴──────────────────────────────────────┘[!WARNING] While Edge runtimes boast zero cold starts, many provider SDKs or vector database clients rely on native Node bindings (like gRPC or OpenSSL). For general AI engineering in Next.js,
runtime = 'nodejs'with explicit streaming headers is the recommended standard.
Common Pitfalls to Avoid
- Forgetting
export const maxDurationon Vercel/AWS: Serverless functions default to 10s or 15s timeouts. LLMs generating 1,000 tokens can easily take 25 seconds. If omitted, your connection abruptly terminates mid-sentence. - Buffering proxies wiping out SSE: If your production architecture uses NGINX, Cloudflare, or an AWS ALB, ensure gzip buffering is disabled for streaming paths by setting
X-Accel-Buffering: noandCache-Control: no-cache. - Omitting
req.signalforwarding: If the client cancels the stream or leaves the page, an unlinked server request keeps generating tokens, racking up unwanted API bill charges!
Interview Angle: How to Answer in System Design
- Question: "When would you choose a Route Handler over a Server Action when implementing an AI assistant?"
- Target Answer:
"I use Route Handlers when implementing conversational streaming interfaces because standard Web Streams (
ReadableStream) over Server-Sent Events cleanly decouple the byte-level transport from the React rendering tree. Route Handlers allow direct propagation of clientAbortSignals to cancel costly GPU inference on the provider side and integrate seamlessly with hooks likeuseChat.Conversely, I reserve Server Actions for deterministic, discrete mutations—like one-shot text categorization, sentiment scoring, or form autofill—where the caller expects a typed Promise, React form state transitions, and automatic cache revalidation."
Quick Check
- Which Next.js config export prevents 10-second serverless timeout kills during long model generations?
- Why is passing
req.signalto upstream providerfetch()calls critical for cost control? - What HTTP header tells reverse proxies like NGINX not to buffer streaming chunks?