Overview
Level 1: Foundations · Module 3: Calling LLM APIs from Next.js · 20 min read
Why this matters
In the fast-moving AI ecosystem, the best model for your product changes every few months. Today it might be Claude 3.7 Sonnet; next month Gemini 2.5 Pro might offer superior context recall; next quarter an open-weights model on Groq or vLLM might slash your inference costs by 80%. If your application code directly couples route handlers, prompt strings, and JSON schemas to a single vendor's SDK, every model transition requires tedious, bug-prone refactoring across dozens of files. Designing a Provider Abstraction Layer with clean interfaces and typed adapters enables your team to switch or A/B test models in production with zero application-level changes.
Learning objectives
- Design a clean
LlmClientinterface isolating vendor-specific SDK quirks. - Implement typed adapters for OpenAI, Anthropic, and Local Mock providers.
- Configure dynamic model routing based on runtime configuration or feature flags.
- Normalize message schemas, tool signatures, and token usage metrics.
- Demonstrate how provider abstractions enable zero-risk model migrations and cost benchmarking.
Architecture: The Adapter Pattern for AI Models
graph TD
Client["Application Layer<br>(Route Handlers, Server Actions)"] -->|"LlmClient.complete(params)"| Abstraction["Provider Abstraction Layer<br>(Unified Interface)"]
subgraph Adapters["Provider Adapters"]
Abstraction --> OpenAIAdapter["OpenAIAdapter<br>(o3-mini, gpt-4o)"]
Abstraction --> AnthropicAdapter["AnthropicAdapter<br>(claude-3-7-sonnet, claude-3-5-haiku)"]
Abstraction --> MockAdapter["MockAdapter<br>(Zero-Cost Local Dev & CI)"]
end
OpenAIAdapter --> OpenAIAPI["OpenAI API"]
AnthropicAdapter --> AnthropicAPI["Anthropic API"]
MockAdapter --> LocalMem["Simulated Memory Stream"]Step 1: Defining the Unified Domain Interface
The core interface must define only what your application needs, entirely decoupled from vendor-specific response envelopes:
// lib/ai/types.ts
export type Role = "system" | "user" | "assistant";
export interface Message {
role: Role;
content: string;
}
export interface CompletionOptions {
messages: Message[];
temperature?: number;
maxTokens?: number;
responseFormat?: "text" | "json";
signal?: AbortSignal;
}
export interface CompletionResult {
content: string;
model: string;
usage: {
promptTokens: number;
completionTokens: number;
totalTokens: number;
};
durationMs: number;
}
export interface StreamChunk {
text: string;
isComplete: boolean;
}
export interface LlmProvider {
name: string;
complete(options: CompletionOptions): Promise<CompletionResult>;
stream(options: CompletionOptions): AsyncGenerator<string, void, unknown>;
}Step 2: Implementing Concrete Provider Adapters
1. OpenAI Adapter
// lib/ai/providers/openai-adapter.ts
import OpenAI from "openai";
import { LlmProvider, CompletionOptions, CompletionResult } from "../types";
export class OpenAIAdapter implements LlmProvider {
name = "openai";
private client: OpenAI;
private defaultModel: string;
constructor(apiKey = process.env.OPENAI_API_KEY, defaultModel = "claude-3-7-sonnet") {
this.client = new OpenAI({ apiKey });
this.defaultModel = defaultModel;
}
async complete(options: CompletionOptions): Promise<CompletionResult> {
const start = performance.now();
const res = await this.client.chat.completions.create(
{
model: this.defaultModel,
messages: options.messages as any,
temperature: options.temperature ?? 0.2,
max_tokens: options.maxTokens ?? 1000,
response_format:
options.responseFormat === "json"
? { type: "json_object" }
: undefined,
},
{ signal: options.signal }
);
return {
content: res.choices[0]?.message?.content || "",
model: res.model,
usage: {
promptTokens: res.usage?.prompt_tokens || 0,
completionTokens: res.usage?.completion_tokens || 0,
totalTokens: res.usage?.total_tokens || 0,
},
durationMs: Math.round(performance.now() - start),
};
}
async *stream(options: CompletionOptions): AsyncGenerator<string, void, unknown> {
const stream = await this.client.chat.completions.create(
{
model: this.defaultModel,
messages: options.messages as any,
stream: true,
temperature: options.temperature ?? 0.2,
},
{ signal: options.signal }
);
for await (const chunk of stream) {
const delta = chunk.choices[0]?.delta?.content || "";
if (delta) yield delta;
}
}
}2. Anthropic Adapter
Notice how Anthropic separates the system prompt into a top-level property, whereas OpenAI includes it in the messages array. The adapter absorbs this discrepancy so your application never has to worry about it:
// lib/ai/providers/anthropic-adapter.ts
import Anthropic from "@anthropic-ai/sdk";
import { LlmProvider, CompletionOptions, CompletionResult } from "../types";
export class AnthropicAdapter implements LlmProvider {
name = "anthropic";
private client: Anthropic;
private defaultModel: string;
constructor(apiKey = process.env.ANTHROPIC_API_KEY, defaultModel = "claude-3-7-sonnet-20250219") {
this.client = new Anthropic({ apiKey });
this.defaultModel = defaultModel;
}
async complete(options: CompletionOptions): Promise<CompletionResult> {
const start = performance.now();
// Split system prompt from conversation messages
const systemMessage = options.messages.find((m) => m.role === "system")?.content;
const conversationMessages = options.messages
.filter((m) => m.role !== "system")
.map((m) => ({
role: m.role as "user" | "assistant",
content: m.content,
}));
const res = await this.client.messages.create(
{
model: this.defaultModel,
max_tokens: options.maxTokens ?? 1000,
system: systemMessage,
messages: conversationMessages,
temperature: options.temperature ?? 0.2,
},
{ signal: options.signal }
);
const firstBlock = res.content[0];
const text = firstBlock?.type === "text" ? firstBlock.text : "";
return {
content: text,
model: res.model,
usage: {
promptTokens: res.usage.input_tokens,
completionTokens: res.usage.output_tokens,
totalTokens: res.usage.input_tokens + res.usage.output_tokens,
},
durationMs: Math.round(performance.now() - start),
};
}
async *stream(options: CompletionOptions): AsyncGenerator<string, void, unknown> {
const systemMessage = options.messages.find((m) => m.role === "system")?.content;
const conversationMessages = options.messages
.filter((m) => m.role !== "system")
.map((m) => ({
role: m.role as "user" | "assistant",
content: m.content,
}));
const stream = await this.client.messages.create(
{
model: this.defaultModel,
max_tokens: options.maxTokens ?? 1000,
system: systemMessage,
messages: conversationMessages,
stream: true,
},
{ signal: options.signal }
);
for await (const chunk of stream) {
if (
chunk.type === "content_block_delta" &&
chunk.delta.type === "text_delta"
) {
yield chunk.delta.text;
}
}
}
}Step 3: Provider Factory with Dynamic Routing
Now you can configure the active provider via an environment variable or feature flag:
// lib/ai/factory.ts
import { LlmProvider } from "./types";
import { OpenAIAdapter } from "./providers/openai-adapter";
import { AnthropicAdapter } from "./providers/anthropic-adapter";
export function getAiProvider(providerName?: string): LlmProvider {
const chosen = providerName || process.env.AI_DEFAULT_PROVIDER || "openai";
switch (chosen) {
case "anthropic":
return new AnthropicAdapter();
case "openai":
default:
return new OpenAIAdapter();
}
}Application Code Stays 100% Clean
Now, when writing a Route Handler or Server Action, you simply write:
// app/api/summarize/route.ts
import { NextRequest, NextResponse } from "next/server";
import { getAiProvider } from "@/lib/ai/factory";
export async function POST(req: NextRequest) {
const { text } = await req.json();
// Swappable without changing a single line of business logic!
const provider = getAiProvider();
const result = await provider.complete({
messages: [
{ role: "system", content: "Summarize the text in 3 bullet points." },
{ role: "user", content: text },
],
});
return NextResponse.json({
summary: result.content,
metrics: { model: result.model, duration: result.durationMs, tokens: result.usage.totalTokens },
});
}Interview Angle: How to Answer in System Design
- Question: "How would you architect an enterprise frontend application to prevent vendor lock-in to OpenAI?"
- Target Answer:
"I implement the Adapter pattern using a normalized domain interface (
LlmProvider). The interface enforces canonical structures for inputs (system prompt, conversation history, sampling parameters) and outputs (standardized token counts, timing metrics, and unified string generators).Concrete adapters for OpenAI, Anthropic, or Mistral handle vendor-specific variances—like Anthropic requiring top-level system prompts or differing SSE event payloads. This allows runtime provider switching via feature flags or fallback routing if the primary vendor suffers an outage."
Quick Check
- Why is the Adapter pattern especially valuable when switching between Anthropic and OpenAI?
- What metric normalization should a provider abstraction perform on raw SDK responses?
- How does this architecture support A/B testing model quality in production?