Overview
Level 3: Advanced · Module 7: Tool Calling, Agents, and MCP · 20 min read · Core Track
Why this matters
By default, Large Language Models are isolated text predictors trapped inside a server box: they cannot check the weather, query your Postgres database, trigger a Stripe refund, or browse the web. Tool Calling (Function Calling) breaks through this sandbox. However, many developers harbor a dangerous misconception: they believe the model itself executes the code. It does not. The model merely emits a structured JSON request declaring intent to run a function. Your backend server parses that intent, verifies authorization, executes the actual code in a secure runtime, and feeds the resulting output back to the model. Understanding the exact 3-legged turn mechanics is the foundation of all agentic engineering.
Learning objectives
- Deconstruct the 3-step Request-Execution-Response tool calling turn.
- Define JSON Schema specifications for model tools.
- Handle model tool call choices:
none,auto,required, or explicit named tool pinning. - Execute tools safely on the server and return observations with correct
tool_call_idlinkage. - Defend tool execution security models in engineering interviews.
The 3-Step Tool Calling Turn Mechanics
sequenceDiagram
autonumber
actor User as User in Browser
participant Server as Next.js Server / Backend
participant LLM as Frontier Model (Claude 3.7 Sonnet / o3-mini)
participant DB as Postgres / External API
User->>Server: "Check inventory for Product #1042"
Server->>LLM: POST /chat/completions (Prompt + Tool Definitions)
Note over LLM: Model detects tool relevance.<br>Halts normal text generation.<br>Emits tool_calls payload!
LLM-->>Server: finish_reason: "tool_calls", args: { productId: "1042" }
Note over Server: Server inspects intent.<br>Executes actual JavaScript / SQL code!
Server->>DB: SELECT stock FROM inventory WHERE id = 1042
DB-->>Server: { stock: 14, warehouse: "Austin" }
Server->>LLM: POST /chat/completions (History + Tool Result Observation)
Note over LLM: Model digests tool observation.<br>Synthesizes human-friendly response.
LLM-->>Server: "Product #1042 currently has 14 units in stock in Austin."
Server-->>User: Renders final answer in UIStep 1: Tool Schema Definition
Tools are registered by passing an array of JSON Schema objects describing the function name, purpose, and typed parameters:
// lib/ai/tools/weather-tool.ts
import { z } from "zod";
export const WeatherParametersSchema = z.object({
city: z.string().describe("The name of the city, e.g. San Francisco"),
unit: z.enum(["celsius", "fahrenheit"]).default("celsius").describe("Temperature unit"),
});
export const weatherToolDefinition = {
type: "function" as const,
function: {
name: "get_current_weather",
description: "Fetches current real-time temperature and weather conditions for a given city.",
parameters: {
type: "object",
properties: {
city: {
type: "string",
description: "The city name, e.g. Paris or Tokyo",
},
unit: {
type: "string",
enum: ["celsius", "fahrenheit"],
description: "Temperature unit preference",
},
},
required: ["city"],
},
},
};Step 2: Server Execution and Response Loop
Here is the complete implementation of the 3-step tool calling loop using the 2026 frontier standard (importing centralized model configuration or using Anthropic Claude 3.7 Sonnet / OpenAI SDK):
// app/api/tool-agent/route.ts
import { Anthropic } from "@anthropic-ai/sdk";
import { DEFAULT_TOOL_CALLING_MODEL } from "@/lib/ai-course/models";
import { weatherToolDefinition } from "@/lib/ai/tools/weather-tool";
const anthropic = new Anthropic({ apiKey: process.env.ANTHROPIC_API_KEY });
// Real code execution handler on server
async function executeTool(name: string, args: Record<string, any>) {
if (name === "get_current_weather") {
// In production, call OpenWeatherMap or internal microservice
return JSON.stringify({
city: args.city,
temperature: 22,
condition: "Sunny with light breeze",
humidity: 45,
});
}
throw new Error(`Unknown tool: ${name}`);
}
export async function POST(req: Request) {
const { prompt } = await req.json();
const messages: Anthropic.MessageParam[] = [
{ role: "user", content: prompt },
];
// 1. Initial turn: Model inspects tools (using 2026 frontier Claude 3.7 Sonnet)
const firstResponse = await anthropic.messages.create({
model: DEFAULT_TOOL_CALLING_MODEL, // "claude-3-7-sonnet"
max_tokens: 1024,
messages,
tools: [
{
name: "get_current_weather",
description: "Fetch live temperature and meteorological conditions for a given city.",
input_schema: {
type: "object",
properties: {
city: { type: "string", description: "City name e.g. San Francisco" },
unit: { type: "string", enum: ["celsius", "fahrenheit"] },
},
required: ["city"],
},
},
],
});
// 2. Check if the model requested tool calls
const toolUseBlock = firstResponse.content.find(
(block): block is Anthropic.ToolUseBlock => block.type === "tool_use"
);
if (toolUseBlock) {
const { name: functionName, input: functionArgs, id: toolUseId } = toolUseBlock;
console.log(`[Tool Execution] Running ${functionName} with:`, functionArgs);
const toolResult = await executeTool(functionName, functionArgs);
// 3. Final turn: Pass assistant response + tool result back to the model
const finalResponse = await anthropic.messages.create({
model: DEFAULT_TOOL_CALLING_MODEL,
max_tokens: 1024,
messages: [
...messages,
{ role: "assistant", content: firstResponse.content },
{
role: "user",
content: [
{
type: "tool_result",
tool_use_id: toolUseId,
content: toolResult,
},
],
},
],
});
const textBlock = finalResponse.content.find((b) => b.type === "text");
return Response.json({
answer: textBlock ? textBlock.text : "",
toolCallsExecuted: 1,
});
}
// Model replied directly without tools
const textBlock = firstResponse.content.find((b) => b.type === "text");
return Response.json({
answer: textBlock ? textBlock.text : "",
toolCallsExecuted: 0,
});
}Cross-Provider Comparison (2026 Edition):
While OpenAI syntax useschat.completions.createwithrole: "tool"andtool_call_id, Anthropic usesmessages.createwithtype: "tool_use"andtype: "tool_result". In production, import the centralized@/lib/ai-course/modelsregistry so frontier models (Claude 3.7 Sonnet, o3-mini, Gemini 2.0 Flash) can be switched with a single edit.
Controlling Tool Choice
You can control the model's tool dispatching behavior via the tool_choice parameter:
tool_choice: "auto"(Default): The model autonomously decides whether to generate plain text or invoke one or more tools.tool_choice: "none": Forces the model to generate plain text only, ignoring all tool definitions.tool_choice: "required": Forces the model to call at least one tool, preventing plain-text conversational evasion.tool_choice: { type: "function", function: { name: "get_current_weather" } }: Forces the model to call that specific function.
Common Pitfalls to Avoid
- Missing
tool_call_idlinkage: When sending arole: "tool"message back to OpenAI or Anthropic, you must supply the exacttool_call_idmatching the initial request. Omitting it results in a400 Bad Request. - Executing unvalidated arguments directly: Always parse arguments through Zod before passing them to internal database queries or shell commands to prevent SQL injection or arbitrary command execution!
- Infinite tool loops: If a tool returns an error string, the model might attempt to call the same failing tool repeatedly. Always cap tool executions with a bounded counter.
Interview Angle: How to Answer in System Design
- Question: "How does tool calling work under the hood, and does the AI model execute our backend code?"
- Target Answer:
"The AI model never executes code. Tool calling is a structured protocol turn:
- We supply JSON Schema definitions of available functions to the model alongside the prompt.
- If the model determines that a tool is required, it stops normal text output and emits a structured JSON payload containing the function name, arguments, and a unique
tool_call_id. - Our server inspects this declaration, validates the parameters against a Zod schema, executes the actual code within our secure backend runtime, and sends the observation back in a
role: 'tool'message. - The model reads this observation in a second forward pass and synthesizes the final user answer."
Quick Check
- Does the LLM directly execute JavaScript or database queries when invoking a tool?
- What attribute links a tool observation message back to the model's initial tool request?
- What happens when
tool_choice: "required"is passed?