Overview
Level 1: Foundations · Module 2: Prompting and Structured Output · 25 min read · Core Track
Why this matters
Frontend apps cannot render free-form conversational prose into charts, data grids, or dynamic forms. If your API returns:
"Sure thing! Here is the user data you asked for: json { "name": "Alice" } Hope this helps!"
Your JSON.parse() will violently crash, blanking your screen with an unhandled exception. To build production-grade web applications on top of AI, you must guarantee that the model returns 100% syntactically valid, schema-compliant JSON on every single invocation.
Learning objectives
- Understand the difference between prompt-based JSON requesting and model-constrained JSON mode.
- Master grammar-constrained sampling (JSON Schema enforcement at the decoding level).
- Validate and parse model outputs with Zod schemas.
- Build a bounded self-healing retry loop when edge-case validation fails.
Concept explanation: Prompting for JSON vs Grammar-Constrained Decoding
Historically, getting JSON required begging the model: "Please return valid JSON only. Do not include markdown ticks. Do not say hello." This failed ~5% of the time, especially when generating long arrays or encountering quotes inside text.
Modern AI providers solve this at the decoding layer through Grammar-Constrained Generation:
graph TD
A["Raw Logits over Vocabulary"] --> B{"Is next token valid under JSON Schema?"}
B -- "No (e.g. invalid syntax or wrong type)" --> C["Logit set to -Infinity<br>(Token masked out)"]
B -- "Yes (valid token adhering to schema)" --> D["Sample Token"]
D --> E["Guaranteed 100% Valid JSON Stream"]When you pass a JSON Schema (via OpenAI's response_format: { type: "json_schema" } or Anthropic's tool-use parameter schema), the inference engine constructs a Context-Free Grammar (CFG) or finite state automaton. During sampling, any token that would violate the JSON specification or your schema is masked out before softmax!
Code walkthrough: End-to-End Extraction with Zod
Here is how to combine Zod with schema-constrained generation in a Next.js Server Action:
// app/actions/extract-ticket.ts
"use server";
import { z } from "zod";
import { Anthropic } from "@anthropic-ai/sdk";
// 1. Define the rigorous Zod schema
export const SupportTicketSchema = z.object({
id: z.string().describe("Unique generated UUID or identifier"),
category: z.enum(["billing", "technical", "feature_request", "general"]),
urgency: z.enum(["low", "medium", "high", "critical"]),
summary: z.string().max(100).describe("Concise 1-sentence summary of the issue"),
actionItems: z.array(z.string()).describe("List of concrete next steps for support"),
estimatedResolutionMinutes: z.number().int().positive(),
});
export type SupportTicket = z.infer<typeof SupportTicketSchema>;
const anthropic = new Anthropic({
apiKey: process.env.ANTHROPIC_API_KEY,
});
/**
* Extracts and strictly validates support ticket data from raw customer chat.
*/
export async function extractTicketAction(rawTranscript: string): Promise<SupportTicket> {
const toolName = "submit_ticket";
// We leverage Tool Calling as a guaranteed schema constraint
const response = await anthropic.messages.create({
model: "claude-3-5-sonnet-20241022",
max_tokens: 1024,
temperature: 0.0,
tools: [
{
name: toolName,
description: "Submit a structured and categorized customer support ticket.",
input_schema: {
type: "object",
properties: {
id: { type: "string" },
category: { type: "string", enum: ["billing", "technical", "feature_request", "general"] },
urgency: { type: "string", enum: ["low", "medium", "high", "critical"] },
summary: { type: "string" },
actionItems: { type: "array", items: { type: "string" } },
estimatedResolutionMinutes: { type: "integer" },
},
required: ["id", "category", "urgency", "summary", "actionItems", "estimatedResolutionMinutes"],
},
},
],
tool_choice: { type: "tool", name: toolName }, // Force model to call this tool
messages: [
{
role: "user",
content: `Extract the support ticket details from this transcript:\n\n${rawTranscript}`,
},
],
});
const toolUseBlock = response.content.find((c) => c.type === "tool_use");
if (!toolUseBlock || toolUseBlock.type !== "tool_use") {
throw new Error("Model failed to invoke extraction tool.");
}
// 2. Runtime Zod validation guarantees full TypeScript type safety
const parseResult = SupportTicketSchema.safeParse(toolUseBlock.input);
if (!parseResult.success) {
console.error("Schema validation failed:", parseResult.error.flatten());
throw new Error("Structured data did not conform to expected schema.");
}
return parseResult.data;
}The Self-Healing Bounded Retry Pattern
What if a smaller or local model produces an invalid enum value? Never crash the page. Implement a bounded repair loop:
export async function extractWithSelfHealing(
rawText: string,
maxAttempts: number = 2
): Promise<SupportTicket> {
let attempt = 0;
let feedback = "";
while (attempt < maxAttempts) {
attempt++;
try {
const prompt = feedback
? `${rawText}\n\nPREVIOUS ERROR: Your previous JSON was invalid:\n${feedback}\nPlease repair the JSON strictly.`
: rawText;
return await extractTicketAction(prompt);
} catch (err: any) {
if (attempt >= maxAttempts) throw err;
feedback = err.message || "Invalid schema formatting.";
}
}
throw new Error("Failed to extract valid data after maximum retries.");
}Common mistakes frontend developers make
- Calling
JSON.parse()without atry/catchblock: Even with JSON mode, network truncation or streaming interruptions can leave partial JSON strings. - Not using Zod
.describe()annotations: Descriptions inside your schema give the model critical semantic clues for what values to assign. - Allowing unconstrained strings when an enum is appropriate: Always use
z.enum([...])rather thanz.string()wherever options are finite (e.g. status, priority, category).
Interview angle: How this appears in interviews
Interviewer: "How do you guarantee that an LLM returns JSON that your React components can render without crashing the frontend?"
Winning Answer:
"I guarantee this through a 3-tier defense:
- Decoding Level: Use provider-level Structured Outputs or Schema-Constrained Tool Calling with
temperature: 0. This forces the inference engine to mask out invalid tokens at the grammar level.- Runtime Validation: Pass the parsed JSON through a strict Zod schema with
safeParse(). If fields are missing or types mismatch, TypeScript types are not assumed.- Graceful Fallbacks & Bounded Retries: If validation fails, trigger a bounded self-healing retry loop passing the Zod error back to the model. If it fails a second time, gracefully degrade to a fallback UI card with human review options."
Quick check
- How does grammar-constrained decoding prevent syntax errors in JSON?
(Answer: It masks out invalid tokens at each generation step so non-conforming tokens cannot be sampled.) - Why is
z.safeParse()preferred overz.parse()when processing LLM outputs?
(Answer:safeParse()returns a success/error object instead of throwing an unhandled runtime error.) - What is the most reliable cross-provider technique to force structured output?
(Answer: Using Tool / Function Calling with a defined JSON schema and forced tool choice.)
Summary
- Unconstrained prompt begging for JSON fails in production; grammar-constrained decoding or tool calling is mandatory.
- Zod provides compile-time and runtime type safety, protecting React components from corrupt state.
- A self-healing retry loop feeds validation errors back to the model for automated error recovery.
- Next up: We will examine Validating Model Output with Zod and Handling Parse Failures.