How the system prompt, message roles, and prompt caching actually shape Claude's output in a real @anthropic-ai/sdk request, covering the CURRENT mid-conversation system-message pattern, structured-output alternatives to the now-deprecated assistant-turn prefill, and the prefix-match caching model that determines whether cache_control actually saves money.
1. What should generally live in the `system` parameter versus `messages`?
Interview Questions
2 questions
Cheatsheet
Download-ready reference
Cheatsheet
SYSTEM vs MESSAGES:
system → stable, request-independent (persona, scope, format rules)
messages → what actually changes per turn (user Qs, assistant replies)
MID-CONVERSATION CONTEXT (current pattern, not editing top-level system):
messages.push({ role: "system", content: "runtime fact..." }), preserves the cached system prefix, non-spoofable operator channel
PROMPT CACHING = PREFIX MATCH, NOT SEMANTIC MATCH:
Any byte difference anywhere in the prefix invalidates everything after it.
Render order: tools → system → messages.
cache_control: { type: "ephemeral" } // 5-min default TTL
cache_control: { type: "ephemeral", ttl: "1h" }
Verify: response.usage.cache_read_input_tokens (should be > 0 on repeats)
response.usage.cache_creation_input_tokens (write cost, ~1.25x)
Silent invalidators: timestamps/UUIDs in system, non-deterministic key
order, varying tool set, model switch, ALL break the whole downstream cache.
STRUCTURED OUTPUT (replaces deprecated assistant-turn prefill, now 400s):
output_config: {
format: { type: "json_schema", schema: { ...JSON Schema... } }
}
COMMON MISTAKES:
❌ interpolating new Date() / a request id into `system`
❌ putting the user's live question inside `system`
❌ relying on an assistant-turn prefill for JSON, now errors
❌ vague system prompts ("be helpful"), inconsistent at scale
Prompt engineering for an application (as opposed to one-off chatting) is really just the discipline of controlling three separate inputs to client.messages.create(), system, messages, and output_config, with enough precision that the model's behavior is predictable across thousands of different end-user inputs, not just the one example you tested by hand.
The system parameter is where persistent, request-independent behavior lives, the model's persona, constraints, and output-format rules that should apply to every turn of a conversation. messages is the actual back-and-forth content, what changes every turn.
const response = await client.messages.create({ model: "claude-opus-4-8", max_tokens: 1024, system: "You are a support assistant for a SaaS billing product. " + "Answer only questions about billing, invoices, and subscriptions. " + "If asked about anything else, say so and redirect the user to general support. " + "Keep responses under 120 words.", messages: [{ role: "user", content: "Why was I charged twice this month?" }],});
A common beginner mistake is cramming everything, persona, constraints, AND the user's actual question, into a single system string that gets rebuilt from scratch on every request. That works, but it throws away prompt caching (below) and makes debugging harder, because there's no clean separation between "what should never change" and "what's different this turn."
Mid-conversation system messages, the current pattern for injected context#
Older patterns for injecting runtime context mid-conversation (a mode switch, a freshly-fetched fact) relied on stuffing a reminder into a user-turn text block. The current, more precise mechanism on supporting models is a role: "system" entry appended directly into messages, rather than editing the top-level system string:
const response = await client.messages.create({ model: "claude-opus-4-8", max_tokens: 1024, system: [ { type: "text", text: STABLE_SYSTEM_PROMPT, cache_control: { type: "ephemeral" } }, ], messages: [ ...priorHistory, { role: "user", content: "What's my order status?" }, { role: "system", content: "The user's account was just flagged for a billing dispute, mention this if relevant." }, ],});
This matters for two reasons: it leaves the top-level system string byte-for-byte unchanged (so the prompt cache built on it is not invalidated, see below), and it carries operator authority rather than being spoofable text a user could have typed themselves, since it arrives as its own distinct role rather than as text embedded inside a user turn.
Prompt caching, a prefix match, not a semantic cache#
cache_control does not mean "cache similar requests." It caches an exact byte-for-byte prefix of the rendered request. Any change anywhere in that prefix, a different timestamp, a reordered JSON key, a different tool list, invalidates the cache for everything after that point.
const response = await client.messages.create({ model: "claude-opus-4-8", max_tokens: 1024, system: [ { type: "text", text: LARGE_SHARED_KNOWLEDGE_BASE_TEXT, // e.g. 20K tokens of product docs cache_control: { type: "ephemeral" }, // 5-minute default TTL }, ], messages: [{ role: "user", content: "How do I set up SSO?" }],});// response.usage.cache_read_input_tokens, served from cache, ~0.1x cost// response.usage.cache_creation_input_tokens, written to cache this request, ~1.25x cost// response.usage.input_tokens, full-price, uncached tokens
The single most common way teams accidentally disable caching entirely is interpolating something that changes every request, new Date(), a request UUID, a user id, directly into the system prompt text. That one interpolation sits early in the byte sequence and invalidates every cache read for the rest of the request, even if 95% of the system prompt is otherwise identical every time. The fix is always the same: keep the frozen, shared portion first (with the cache_control breakpoint at its end), and put anything that varies per-request or per-user after that breakpoint, in messages instead.
Replacing assistant-turn prefills with structured outputs#
An older prompting trick forced the response into a particular shape by pre-filling the start of the assistant's turn (e.g. ending your messages array with { role: "assistant", content: '{"name": "' } to force JSON). This is deprecated on current-generation models and returns a 400 error. The current, more robust replacement is output_config.format:
Structured outputs guarantee the response actually validates against your schema, a stronger guarantee than a prefill ever gave you, since a prefill only nudged the model's starting tokens without enforcing the rest of the structure.
Predict which of these two system prompts produces more consistent behavior across many different user inputs, before checking the solution.
// Prompt Asystem: "Be helpful and answer questions about our product."// Prompt Bsystem: "You are a support assistant for Acme's billing product. " + "Scope: billing, invoices, subscriptions only. " + "If the question is out of scope, say: 'I can only help with billing questions, " + "for anything else, contact support@acme.com.' " + "Response length: under 100 words, no markdown headers."
Which one is more likely to produce predictable, on-brand behavior at scale, and why?
Solution
Prompt B, it explicitly defines scope, gives a literal fallback response for out-of-scope questions, and constrains format and length. Prompt A's vagueness ("be helpful") leaves the model to infer scope and tone per-request, which means two different users asking similar questions can get meaningfully different treatment (one gets a two-paragraph markdown-formatted essay, another gets a one-liner) purely based on how the model happened to interpret "helpful" for that particular phrasing. Precise system prompts trade a bit of upfront writing effort for consistency across the thousands of real user inputs a vague prompt would otherwise handle inconsistently.
Build a tiny prompt-template helper that enforces the separation between stable, cacheable system content and per-request variable content, the actual discipline that makes caching work in practice.
interface PromptTemplate { stableSystem: string; // never changes across calls, safe to cache buildMessages: (userInput: string, context?: string) => Anthropic.MessageParam[];}function createSupportPrompt(): PromptTemplate { const stableSystem = "You are a support assistant for Acme's billing product. " + "Scope: billing, invoices, subscriptions only. Response length: under 100 words."; return { stableSystem, buildMessages(userInput, context) { const messages: Anthropic
This is the actual mechanism real prompt-management systems use: a fixed, versioned, cacheable "core" prompt plus a clearly separated channel for runtime-varying context, never one giant string rebuilt from scratch per request.
Prompt caching's prefix-match behavior is a direct consequence of how the request is serialized and hashed before being sent, reordering a JSON object's keys or changing a single character anywhere in the prefix produces different bytes, the same non-negotiable rule covered generally in Generics in TypeScript around structural type matching being exact, not fuzzy. The mid-conversation role: "system" message pattern exists specifically because text embedded in a user-role turn is otherwise indistinguishable from anything the end user typed, which is the same class of trust-boundary problem covered in Cross-Site Scripting (XSS), untrusted input and trusted instructions must stay in distinguishable channels, not collapsed into one string.
1. Interpolating a timestamp or request ID into the system prompt#
// ❌ invalidates prompt caching on every single requestsystem: `Current time: ${new Date().toISOString()}\n\nYou are a helpful assistant...`
Every request now has a unique prefix, so cache_read_input_tokens will be zero forever, even though the actual instructions never change. Move genuinely dynamic values (like a timestamp) into messages, after the cache breakpoint, or drop them if they aren't load-bearing for the model's behavior.
2. Cramming the user's actual question into the system prompt#
// ❌ system prompt now changes every request, the whole point of `system` is lostsystem: `You are a helpful assistant. The user asked: "${userQuestion}"`
This defeats caching (system changes every call) and blurs the model's understanding of what's an instruction versus what's user content. The user's question belongs in messages as a user-role turn; system should describe stable behavior only.
3. Relying on an assistant-turn prefill for structured output#
// ❌ 400 error on current-generation modelsmessages: [ { role: "user", content: "Extract the name and plan." }, { role: "assistant", content: '{"name": "' },]
Assistant-turn prefills are deprecated on current-generation models and now return a 400 error. Use output_config: { format: { type: "json_schema", schema: {...} } } instead, it's both supported and gives you an actual validation guarantee the prefill trick never did.
Keep system for stable, request-independent behavior only, persona, scope, format rules. Put anything that changes per-request in messages.
Put a cache_control breakpoint at the end of the stable portion of system (or the last shared block of a long messages prefix) so repeated requests actually hit the cache.
Use a role: "system" message in messages for context that needs to arrive mid-conversation, instead of editing the top-level system string or embedding it in a user turn.
Verify caching is actually working by checking response.usage.cache_read_input_tokens on repeated requests, if it's zero, something in the prefix is varying and needs to be found and moved.
A cache breakpoint requires a minimum prefix length (model-dependent, roughly 1024, 4096 tokens), a short system prompt below that threshold silently won't cache at all, with no error.
Prompt caching's write cost (~1.25x for the default 5-minute TTL) only pays off after roughly two requests reuse the same prefix, don't bother caching a system prompt that's only ever sent once.
, content:
"Jane Doe is on the Pro plan."
}],
});
.
MessageParam
[]
=
[{ role:
"user"
, content: userInput }];
// Per-request, non-cacheable context goes in `messages`, appended AFTER
// the stable system prompt's cache breakpoint, never interpolated
messages: template.buildMessages("Why was I charged twice?", "Account flagged for dispute."),
});
Replace assistant-turn prefills with output_config.format for anything that needs a guaranteed shape (JSON, an enum-constrained field).
Be explicit about scope and fallback behavior in the system prompt (what to say when a question is out of scope) rather than leaving it implicit, implicit scope produces inconsistent behavior across real user inputs.