Module 11 · AI Agents & MCP · Deep Dive
In Module 6 you built RAG — retrieve, stuff context, answer. Here you cross the line most candidates never cross: an LLM that can decide to call tools, read the results, and act. That's an agent.
BasicIntermediateBuild
Three things sound similar but aren't. Pin the difference and everything else follows.
| Shape | What happens | Who decides the steps |
|---|---|---|
| Plain LLM call | One request, one response. | Nobody — it just answers. |
| RAG (Module 6) | You retrieve chunks, stuff them into the prompt, then call the model once. | You, in fixed code. |
| Agent | The model runs in a loop: it may ask to call a tool, you run it, you hand back the result, it continues — until it answers. | The model, at runtime. |
An agent is just an LLM in a loop that can call tools and act on the results. That's the whole idea. The model doesn't run your code — it emits a request to run a tool, your harness runs it, and you feed the answer back. The "intelligence" is the model choosing which tool and when; the "muscle" is your code.
RAG bridge: in basic RAG you always retrieve, even when the question doesn't need it. An agent retrieves only when it decides retrieval helps — and can search twice, or not at all.Every agent — Claude Code, a customer-support bot, your DocChat agent — runs the same loop. Memorise this; it's the spine of the module.
prompt ─▶ model
│
├─ stop_reason == "end_turn" ─▶ done, return the text
│
└─ stop_reason == "tool_use"
│ model emitted one or more tool_use blocks
▼
you run each tool
│
▼
append a tool_result block per tool_use
│
└─▶ call the model again ──▶ (back to top)
The model never sees your database or your filesystem. It sees tool definitions, decides to call one, and pauses. You execute, return a result, and it resumes with that knowledge in context. Loop until stop_reason is "end_turn".
messages list every turn, growing it with each assistant response and each tool result. Lose track of that list and the model loses its memory.
Before tools, the bare call. Install the official SDK and set your key:
pip install anthropic
export ANTHROPIC_API_KEY="sk-ant-..."
first_call.py
import anthropic client = anthropic.Anthropic() # reads ANTHROPIC_API_KEY resp = client.messages.create( model="claude-sonnet-4-6", # balanced default for an agent loop max_tokens=1024, system="You are DocChat, a precise assistant for company documents.", messages=[ {"role": "user", "content": "What can you help me with?"}, ], ) # content is a LIST of blocks, not a string — check .type before .text for block in resp.content: if block.type == "text": print(block.text)
Four things to internalise: model is a current ID (more below), max_tokens is the hard output cap, system sets the persona, and messages is the conversation. The response content is a list of blocks — text now, but soon tool_use blocks too.
claude-opus-4-8 (most capable), claude-sonnet-4-6 (balanced — good default for agent loops), claude-haiku-4-5 (fast/cheap). Never guess a date suffix or use retired names like claude-3. OpenAI's GPT models are a valid alternative provider with a similar tool-calling shape, but we centre on Claude here.
A tool is a JSON description you hand the model. It has three parts: a name, a description, and an input_schema (JSON Schema for the arguments). The model reads these and decides when to call.
search_tool = {
"name": "search_documents",
"description": (
"Search the DocChat knowledge base for passages relevant to a "
"query. Use this whenever the user asks about the contents of "
"company documents. Returns the top matching chunks with their "
"document titles."
),
"input_schema": {
"type": "object",
"properties": {
"query": {
"type": "string",
"description": "The natural-language search query.",
},
"top_k": {
"type": "integer",
"description": "How many chunks to return (default 5).",
},
},
"required": ["query"],
},
}
When the model wants a tool, the response comes back with stop_reason == "tool_use" and a tool_use block in content:
# A tool_use block the model emitted ToolUseBlock( type="tool_use", id="toolu_01A...", # you MUST echo this id back name="search_documents", input={"query": "refund policy", "top_k": 5}, )
You run the tool, then reply with a tool_result block in a new user message. The tool_use_id must match the block's id exactly — that's how the model pairs result to request:
{
"role": "user",
"content": [
{
"type": "tool_result",
"tool_use_id": "toolu_01A...",
"content": "[1] refund-policy.pdf: Refunds within 30 days...",
}
],
}
response.content (the tool_use blocks) to messages before you add the result — drop it and the API rejects the next call. (2) If the model emits several tool_use blocks at once, return all their tool_result blocks in a single user message. Splitting them across messages quietly trains the model to stop calling tools in parallel.
Now the whole thing in one place. This is the manual loop — you own every step, which is exactly what you want for an agent you control (logging, validation, approval gates).
agent_loop.py
import anthropic client = anthropic.Anthropic() def run_tool(name, tool_input): if name == "search_documents": return search_documents(**tool_input) # your real impl return f"Unknown tool: {name}" def agent(user_message, tools, max_steps=6): messages = [{"role": "user", "content": user_message}] for _ in range(max_steps): # GUARDRAIL: cap iterations resp = client.messages.create( model="claude-sonnet-4-6", max_tokens=1024, system="You are DocChat. Ground every answer in retrieved passages.", tools=tools, messages=messages, ) # Model is done — return the final text if resp.stop_reason == "end_turn": return next(b.text for b in resp.content if b.type == "text") # Model wants tools — append its turn, then run each tool messages.append({"role": "assistant", "content": resp.content}) results = [] for block in resp.content: if block.type == "tool_use": output = run_tool(block.name, block.input) results.append({ "type": "tool_result", "tool_use_id": block.id, "content": output, }) messages.append({"role": "user", "content": results}) return "Stopped: hit the step limit without a final answer."
Read it as the loop diagram from §2 made real: call → check stop_reason → if tools, run them and feed back → repeat. The max_steps cap is your first guardrail (more in §9).
client.beta.messages.tool_runner) that drives this loop for you. Learn the manual version first — when you need approval gates or custom logging, you'll be glad you can hand-roll it.
Sometimes you don't want prose — you want a typed object you can parse. Two clean ways on Claude.
(a) Structured outputs — constrain the whole response to a JSON Schema with output_config. The first text block is guaranteed valid JSON:
resp = client.messages.create(
model="claude-sonnet-4-6",
max_tokens=512,
messages=[{"role": "user",
"content": "Extract title and page count from this doc..."}],
output_config={
"format": {
"type": "json_schema",
"schema": {
"type": "object",
"properties": {
"title": {"type": "string"},
"pages": {"type": "integer"},
},
"required": ["title", "pages"],
"additionalProperties": False,
},
}
},
)
import json
data = json.loads(next(b.text for b in resp.content if b.type == "text"))
(b) Tool-based — define a tool whose input_schema is the shape you want, set strict: true, and the model's tool_use.input is your validated object. This doubles as the pattern for forcing a classification label (an enum field of valid labels).
{" trick returns a 400. Use output_config.format or a tool schema instead — that's the modern, supported way to force shape.
The senior instinct is knowing when not to build an agent. More autonomy means more cost, more latency, and more ways to go wrong. Match the tool to the job.
| Task | Reach for | Why |
|---|---|---|
| Classify, summarise, extract one thing | Plain LLM call | One request is enough; an agent adds cost for nothing. |
| "Answer from my docs" with fixed retrieval | RAG | You know you always retrieve. No decision needed. |
| Multi-step pipeline, you control the order | Workflow (code orchestrates calls) | Predictable, testable, cheap. Most "agent" use cases are really this. |
| Open-ended task where the model must decide its own path | Agent (tool loop) | Worth the cost only when steps can't be fixed in advance. |
An agent loop without guardrails is a way to spend money in an infinite loop. Wire these in from the start:
max_steps cap from §6. A confused model can call tools forever; the cap stops the bleeding.tool_use.input is untrusted. Never interpolate it straight into SQL or a shell command. Validate, parametrise, sanitise — treat it like user input, because it effectively is.claude-haiku-4-5 for simple tool routing, claude-sonnet-4-6 for most agents, claude-opus-4-8 only when reasoning depth pays off.search_documents tool (queries DocChat's pgvector store) and a get_document_metadata tool. The model decides when to search, reads the results, can look up metadata, and answers a grounded question — agentic RAG, not fixed RAG.
app/agent_rag.py
from fastapi import APIRouter from pydantic import BaseModel import anthropic from .store import vector_search, fetch_metadata # your Module 6 pgvector code router = APIRouter() client = anthropic.Anthropic() TOOLS = [ { "name": "search_documents", "description": ( "Search the DocChat pgvector store for passages relevant to a " "query. Use this whenever the user asks about document contents." ), "input_schema": { "type": "object", "properties": { "query": {"type": "string", "description": "Search query."}, }, "required": ["query"], "additionalProperties": False, }, }, { "name": "get_document_metadata", "description": ( "Look up a document's title, author, and page count by its id. " "Use this when the user asks about a specific document rather " "than its contents." ), "input_schema": { "type": "object", "properties": { "doc_id": {"type": "string", "description": "The document id."}, }, "required": ["doc_id"], "additionalProperties": False, }, }, ] def run_tool(name, args): if name == "search_documents": hits = vector_search(args["query"], top_k=5) return "\n".join(f"[{h.doc_id}] {h.title}: {h.text}" for h in hits) if name == "get_document_metadata": m = fetch_metadata(args["doc_id"]) return f"title={m.title}, author={m.author}, pages={m.pages}" return f"Unknown tool: {name}" class Ask(BaseModel): question: str @router.post("/agent/ask") def ask(body: Ask): messages = [{"role": "user", "content": body.question}] for _ in range(6): # max-iteration guardrail resp = client.messages.create( model="claude-sonnet-4-6", max_tokens=1024, system=("You are DocChat. Decide whether to search the corpus or " "look up metadata, then answer grounded in what you find. " "Cite document titles."), tools=TOOLS, messages=messages, ) if resp.stop_reason == "end_turn": answer = next(b.text for b in resp.content if b.type == "text") return {"answer": answer} messages.append({"role": "assistant", "content": resp.content}) results = [ {"type": "tool_result", "tool_use_id": b.id, "content": run_tool(b.name, b.input)} for b in resp.content if b.type == "tool_use" ] messages.append({"role": "user", "content": results}) return {"answer": "I couldn't finish within the step limit."}
Ask it "What's our refund window, and who wrote the policy doc?" and watch the loop: the model searches for the refund passage, then calls get_document_metadata for the author — two tools, one question, all decided by the model. That's the leap past plain RAG.
Answer from memory — retrieval is what moves "I read it" to "I know it".
What best defines an "agent"?
What stop_reason means the model wants a tool?
How do you return a tool's output to the model?
What most decides whether the model calls a tool?
Which guardrail prevents an infinite agent loop?
tool_use / tool_result loop, and tool_choice. Pair it with Structured outputs for the JSON-forcing patterns.