Overview
Level 1: Foundations · Module 3: Calling LLM APIs from Next.js · 18 min read
Why this matters
During local frontend development and continuous integration (CI) test runs, calling live LLM APIs is an anti-pattern: it burns corporate budgets, hits rate limits during test suites, and introduces non-deterministic test flakiness. Moreover, improper handling of API keys across .env, preview branches, and deployment targets poses severe security vulnerabilities. By implementing strict zero-cost local mocks and secure secrets hygiene, frontend teams can build, test, and iterate on complex AI interfaces completely offline at lightning speed.
Learning objectives
- Enforce strict environment variable validation at build time using
t3-envor Zod. - Build a high-fidelity
MockLlmAdaptersimulating realistic token streaming delays. - Configure deterministic fixture replay for Vitest and Playwright test suites.
- Manage secrets safely across local, preview, and production environments.
- Prevent credential leaks with pre-commit hooks and git secret scanners.
Architecture: Secrets Validation & Development Mocks
graph TD
subgraph EnvBoot["Application Boot (env.ts)"]
Env[".env.local"] --> Zod["Zod Validation Schema"]
Zod -->|"Missing Key in Prod"| Fatal["Throw Build Error immediately!"]
Zod -->|"MOCK_AI=true or Dev"| MockMode["Route to Mock Provider"]
Zod -->|"Valid Keys Present"| ProdMode["Route to Live Provider"]
end
subgraph Runtime["Runtime Execution"]
MockMode --> MockAdapter["MockLlmAdapter<br>(Deterministic, 0$ Cost, 0ms TTFT)"]
ProdMode --> LiveAdapter["OpenAI / Anthropic"]
endStep 1: Validating Secrets at App Boot
Do not access process.env.OPENAI_API_KEY directly inside individual route handlers without validation. Instead, define a centralized environment schema that fails early during build:
// lib/env.ts
import { z } from "zod";
const envSchema = z.object({
NODE_ENV: z.enum(["development", "test", "production"]).default("development"),
// Mock flag allows developers to run the entire app without any paid API keys
MOCK_AI: z
.string()
.transform((val) => val === "true" || val === "1")
.default("false"),
OPENAI_API_KEY: z.string().min(1).optional(),
ANTHROPIC_API_KEY: z.string().min(1).optional(),
});
const _env = envSchema.safeParse(process.env);
if (!_env.success) {
console.error("❌ Invalid environment variables:", _env.error.format());
throw new Error("Invalid environment configuration.");
}
// In production, enforce that at least one provider key is supplied
if (_env.data.NODE_ENV === "production" && !_env.data.MOCK_AI) {
if (!_env.data.OPENAI_API_KEY && !_env.data.ANTHROPIC_API_KEY) {
throw new Error("❌ Production deployments must supply at least one AI provider API key.");
}
}
export const env = _env.data;Step 2: High-Fidelity Streaming Mock Adapter
A good mock is not a static string; it realistically mimics SSE chunks, artificial latency, and token pacing so you can test your loading states, auto-scrolling, and stop buttons without paying a cent:
// lib/ai/providers/mock-adapter.ts
import { LlmProvider, CompletionOptions, CompletionResult } from "../types";
export class MockLlmAdapter implements LlmProvider {
name = "mock";
async complete(options: CompletionOptions): Promise<CompletionResult> {
const userPrompt = options.messages[options.messages.length - 1]?.content || "";
// Simulate realistic 300ms network roundtrip
await new Promise((r) => setTimeout(r, 300));
const mockResponse = options.responseFormat === "json"
? JSON.stringify({ summary: "Mock JSON extracted successfully", confidence: 0.98 })
: `[Mock AI Response]: Analysis complete for: "${userPrompt.slice(0, 30)}..."`;
return {
content: mockResponse,
model: "mock-llm-v1",
usage: { promptTokens: 25, completionTokens: 40, totalTokens: 65 },
durationMs: 300,
};
}
async *stream(options: CompletionOptions): AsyncGenerator<string, void, unknown> {
const mockWords = [
"This", " is", " a", " high-fidelity", " mock", " streaming",
" token", " generator.", " It", " allows", " frontend", " developers",
" to", " test", " streaming", " UIs,", " cancellation,", " and",
" animations", " without", " incurring", " any", " API", " costs."
];
for (const word of mockWords) {
if (options.signal?.aborted) {
throw new DOMException("Mock generation aborted", "AbortError");
}
// Simulate realistic 40ms token pacing (~25 tokens/sec)
await new Promise((r) => setTimeout(r, 40));
yield word;
}
}
}Step 3: Wire into the Provider Factory
// lib/ai/factory.ts
import { env } from "@/lib/env";
import { LlmProvider } from "./types";
import { OpenAIAdapter } from "./providers/openai-adapter";
import { MockLlmAdapter } from "./providers/mock-adapter";
export function getAiProvider(): LlmProvider {
// If MOCK_AI=true or no keys configured in development, use the mock engine
if (env.MOCK_AI || (!env.OPENAI_API_KEY && env.NODE_ENV === "development")) {
console.info("⚡ Using MockLlmAdapter (0 API cost)");
return new MockLlmAdapter();
}
return new OpenAIAdapter(env.OPENAI_API_KEY);
}Best Practices for Team Secret Hygiene
- Commit
.env.example, Never.env.local: Ensure.env.exampledocuments all required keys with dummy values (OPENAI_API_KEY=sk-proj-...). - Setup Pre-commit Hooks with GitLeaks or Trivy: Install automated secret scanning to prevent accidental commits of
sk-...strings before they reach GitHub. - Use Scoped Team Keys: Generate provider keys with spending limits (e.g. $20/month budget ceiling) for junior developers and staging environments.
- Isolate CI Workflows: Automated test runs (
npm run test,playwright test) must always run withMOCK_AI=trueto guarantee zero flaky network failures and zero cloud spend.
Interview Angle: How to Answer in System Design
- Question: "How do you test AI features in CI/CD without burning budget or risking rate limits?"
- Target Answer:
"We decouple the UI and route handler layers from live model endpoints using a provider adapter interface. In CI and local development, we set
MOCK_AI=true, which injects aMockLlmAdapter. This adapter yields realistic token chunks via an async generator, reproducing network delays, streaming backpressure, and abort signals deterministically.For integration testing, we use pre-recorded golden fixture snapshots rather than non-deterministic live calls, ensuring 100% reproducible tests with zero cloud expenditure."
Quick Check
- Why should CI/CD test pipelines never hit live provider API endpoints?
- What is the benefit of simulating token pacing (e.g. 40ms per chunk) in a local mock?
- Which git scanning tool catches accidentally committed API keys before they hit origin?