Overview
Level 4: Production & Interviews · Module 17: LLMOps, Gateways, and Production Deployment · 20 min read · Extended Track
Last updated: November 2026
Production generative AI architectures typically consist of a two-tier topology:
- Frontend / Edge Layer (Next.js): Serves React Server Components, handles user authentication, renders optimistic streaming chat interfaces, and streams Server-Sent Events.
- AI Inference & Worker Layer (FastAPI / Celery): Runs Python microservices for LangGraph state machine execution, PDF extraction, vector embeddings, and GPU inference.
Packaging both tiers into hardened, minimal, reproducible Docker images is essential for zero-downtime Kubernetes and cloud deployments.
1. Two-Tier AI Application Architecture
graph LR
Browser["User Browser"] --> NextContainer["Next.js Container<br>(Node.js Alpine / Standalone)"]
NextContainer --> PythonContainer["FastAPI AI Container<br>(Python Slim + uv)"]
PythonContainer --> Redis["Redis Queue / Cache"]
PythonContainer --> VectorDB["pgvector Postgres DB"]2. Multi-Stage Next.js Standalone Dockerfile
To minimize image size from 1.2 GB to under 90 MB, we leverage Next.js output: "standalone" which bundles only the required node_modules:
# 1. Base stage
FROM node:22-alpine AS base
RUN apk add --no-cache libc6-compat
WORKDIR /app
# 2. Dependencies stage
FROM base AS deps
COPY package.json package-lock.json* ./
RUN npm ci
# 3. Builder stage
FROM base AS builder
WORKDIR /app
COPY --from=deps /app/node_modules ./node_modules
COPY . .
ENV NEXT_TELEMETRY_DISABLED=1
ENV NODE_ENV=production
RUN npm run build
# 4. Production Runner stage
FROM base AS runner
WORKDIR /app
ENV NODE_ENV=production
ENV PORT=3000
ENV HOSTNAME="0.0.0.0"
# Non-root security user
RUN addgroup --system --gid 1001 nodejs && \
adduser --system --uid 1001 nextjs
COPY --from=builder /app/public ./public
# Copy standalone build output
COPY --from=builder --chown=nextjs:nodejs /app/.next/standalone ./
COPY --from=builder --chown=nextjs:nodejs /app/.next/static ./.next/static
USER nextjs
EXPOSE 3000
CMD ["node", "server.js"]3. Hardened FastAPI Python Dockerfile with uv
FROM python:3.12-slim-bookworm AS builder
# Install uv binary
COPY --from=ghcr.io/astral-sh/uv:latest /uv /bin/uv
WORKDIR /app
ENV UV_COMPILE_BYTECODE=1
ENV UV_LINK_MODE=copy
# Install dependencies first for Docker layer caching
COPY pyproject.toml uv.lock ./
RUN uv sync --frozen --no-install-project --no-dev
# Copy application source
COPY . .
RUN uv sync --frozen --no-dev
# Final slim production runner
FROM python:3.12-slim-bookworm AS runner
WORKDIR /app
# Security: Run as non-root user
RUN useradd -u 1001 -m -s /bin/bash appuser
COPY --from=builder --chown=appuser:appuser /app/.venv /app/.venv
COPY --from=builder --chown=appuser:appuser /app ./
USER appuser
ENV PATH="/app/.venv/bin:$PATH"
ENV PYTHONUNBUFFERED=1
EXPOSE 8000
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000", "--workers", "4"]4. Docker Compose for Local Full-Stack Orchestration
version: "3.8"
services:
web:
build:
context: .
dockerfile: Dockerfile.nextjs
ports:
- "3000:3000"
environment:
- AI_BACKEND_URL=http://ai-service:8000
- NEXTAUTH_SECRET=local-development-secret
depends_on:
- ai-service
ai-service:
build:
context: ./backend
dockerfile: Dockerfile.python
ports:
- "8000:8000"
environment:
- REDIS_URL=redis://redis:6379/0
- ANTHROPIC_API_KEY=${ANTHROPIC_API_KEY}
depends_on:
- redis
redis:
image: redis:7-alpine
ports:
- "6379:6379"
volumes:
- redis_data:/data
volumes:
redis_data:5. Interview Angle
Staff Engineer Question: "How do you prevent zombie worker processes and ensure graceful shutdown during Kubernetes pod rolling updates when streaming active LLM completions?"
Principal Answer: In container environments, Kubernetes sends a
SIGTERMbefore terminating a pod. In our Uvicorn/FastAPI configuration, we set--timeout-graceful-shutdown 60. Upon receivingSIGTERM, the reverse proxy stops routing new incoming requests, but allows ongoing Server-Sent Events (SSE) connections up to 60 seconds to finish transmitting tokens. In Next.js, we attach anAbortControllersignal to the upstream fetch; if the client disconnects or the pod enters termination, the abort signal propagates immediately, canceling GPU token generation and stopping unnecessary billing.