AI Agent Design Patterns Every Engineer Should Know

Software engineering has design patterns. AI agent engineering has them too. The difference is that most teams rediscover them independently, in production, under pressure.

This reference documents the essential patterns that structure well-built agentic systems. These aren’t theoretical constructs. They’re reusable solutions to recurring problems that appear when you move agents from notebooks into production: tool overreach, uncontrolled loops, inconsistent outputs, slow chains of sequential calls, and catastrophic failures from unvalidated tool use.

Read this before you architect your next agent. Or read it to name things you’ve already built.

Why Design Patterns Matter for Agents

LLM-based agents have a unique failure mode: the model is both the brain and the source of nondeterminism. When behavior diverges from expectations, you can’t just read a stack trace. You need patterns that make the system’s intent legible, its failure modes predictable, and its components independently testable.

Design patterns for agents serve three goals:

Composability: Break complex behavior into parts that can be developed, tested, and replaced independently.
Observability: Create natural seams for logging, tracing, and monitoring.
Safety: Encode constraints at the architectural level, not just the prompt level.

Core Patterns

ReAct: Reasoning + Action Loop

Intent: Interleave LLM reasoning steps with tool calls in a continuous loop until the goal is achieved.

ReAct (Yao et al., 2022) is the foundational agent loop. The model produces a Thought (internal reasoning), selects an Action (a tool call), receives an Observation (the tool result), then reasons again. This continues until the model concludes the task is done.

Thought: I need to find the current price of AAPL.
Action: search_web(query="AAPL stock price")
Observation: AAPL is trading at $211.47 as of market close.
Thought: I have the price. I can now answer the user.
Action: finish(answer="AAPL closed at $211.47.")

When to use it: Tasks where the next step can’t be fully determined upfront, e.g., multi-hop research, dynamic form-filling, or debugging pipelines.

Failure mode: Infinite reasoning loops. If a tool returns ambiguous output, the model can issue the same action repeatedly. Always enforce a max_steps guard.

Orchestrator-Worker

Intent: A central orchestrator LLM decomposes a goal into sub-tasks and delegates each to a specialized worker agent.

The orchestrator never does the work. It plans, assigns, monitors, and synthesizes. Workers are purpose-built agents with narrow tool access and a focused system prompt.

graph TD
    User["User Goal"] --> Orch["Orchestrator Agent"]
    Orch --> W1["Worker: Web Research Agent"]
    Orch --> W2["Worker: Data Analysis Agent"]
    Orch --> W3["Worker: Report Writer Agent"]
    W1 --> Orch
    W2 --> Orch
    W3 --> Orch
    Orch --> Final["Synthesized Answer"]

When to use it: Complex tasks that require heterogeneous skills, parallel execution, or domain-specific context isolation. This is the dominant pattern in enterprise agent platforms like LangGraph multi-agent graphs and AutoGen group chats.

Critical rule: Workers should be stateless. Pass all required context from the orchestrator explicitly rather than relying on shared global state. This keeps workers independently testable.

Failure mode: Orchestrator prompt bloat. If the orchestrator’s context grows unbounded as it receives results from workers, token costs spike and model behavior degrades. Summarize completed worker outputs before feeding them back.

Evaluator-Optimizer

Intent: A second LLM instance evaluates the output of the primary agent and either accepts it, provides critique for revision, or rejects it entirely.

This is Reflexion (Shinn et al., 2023) generalized as a pattern. The agent produces a draft; the evaluator scores it against a rubric; the agent revises. Repeat until the evaluator passes it.

graph LR
    Goal --> Agent
    Agent -->|Draft| Evaluator
    Evaluator -->|Pass| Output
    Evaluator -->|Fail + Critique| Agent

The evaluator can be:

A second call to the same model with a different prompt focused on critique.
A smaller, faster model (e.g., GPT-4o mini) configured to output a structured {score, issues, suggestions} object.
A deterministic test suite (for code generation: run the tests, feed compiler errors back as the “critique”).

When to use it: Code generation, structured data extraction, technical writing, or any task where quality has an objective or semi-objective definition.

Production note: Use a structured output schema for the evaluator’s response. If the evaluator is free-form prose, the revision loop becomes unpredictable. Pydantic makes this straightforward:

from pydantic import BaseModel
from typing import Literal, List

class EvaluationResult(BaseModel):
    verdict: Literal["pass", "revise", "reject"]
    score: float  # 0.0 to 1.0
    issues: List[str]
    suggestions: List[str]

Failure mode: Stuck revision loops. If the agent can’t satisfy the evaluator’s criteria (e.g., the goal is contradictory), the loop runs to max_iterations. Set a hard cap and surface the best-so-far result rather than failing silently.

Parallel Fan-Out

Intent: Decompose a task into independent sub-tasks, execute them concurrently, and merge the results.

This is the core latency optimization pattern. Sequential chains are the single biggest performance killer in agent systems. If three data sources need querying, there’s no reason to query them one by one.

graph TD
    Start["Decompose Goal"] --> T1["Task A"]
    Start --> T2["Task B"]
    Start --> T3["Task C"]
    T1 --> Merge["Aggregate Results"]
    T2 --> Merge
    T3 --> Merge
    Merge --> Synthesize["LLM: Synthesize Final Answer"]

Python’s asyncio with gather handles this cleanly:

import asyncio

async def fetch_all(queries: list[str]) -> list[str]:
    tasks = [search_web(q) for q in queries]
    return await asyncio.gather(*tasks)

results = asyncio.run(fetch_all([
    "Q1 revenue AAPL 2025",
    "Q1 revenue MSFT 2025",
    "Q1 revenue GOOG 2025",
]))

When to use it: Any task that can be expressed as a DAG (Directed Acyclic Graph) of sub-tasks with dependencies. LLM Compiler (Kim et al., 2023) formalizes this into an LLM-generated DAG.

Failure mode: Resource contention. Unbounded fan-out can saturate API rate limits. Use a semaphore to cap concurrent calls:

sem = asyncio.Semaphore(5)

async def bounded_fetch(query: str) -> str:
    async with sem:
        return await search_web(query)

Tool Router

Intent: A classification step selects which tool (or sub-agent) to invoke before the main execution chain runs.

Without routing, an agent with 30 tools has to reason about all 30 at every step. This is expensive and error-prone. The router pattern inserts a lightweight classification call upfront that narrows the tool surface before the main agent runs.

Routing can be implemented at three levels of sophistication:

Level	Mechanism	Latency	Accuracy
Rule-Based	Regex / keyword matching	~0ms	Low
Classifier Model	Small fine-tuned model	~50ms	High
LLM Classification	GPT-4o mini with structured output	~200ms	Very High

A production router using structured output:

from enum import Enum
from pydantic import BaseModel

class ToolCategory(str, Enum):
    web_search = "web_search"
    code_execution = "code_execution"
    database_query = "database_query"
    document_retrieval = "document_retrieval"

class RouterDecision(BaseModel):
    category: ToolCategory
    confidence: float
    reasoning: str

When to use it: Any agent with more than 5-7 tools, or when routing to specialized sub-agents.

Failure mode: Router hallucination. If the classifier is wrong, the entire execution chain runs with the wrong tools. Log routing decisions and monitor the distribution. A sudden shift in routing distribution often signals prompt drift or data distribution shift.

Human-in-the-Loop (HITL)

Intent: The agent pauses execution at defined checkpoints and requests human review or approval before continuing.

Not every action should be autonomous. Irreversible operations (sending emails, executing database writes, calling payment APIs) carry real-world consequences. The HITL pattern encodes this as a first-class architectural concern rather than an afterthought.

There are three implementation levels:

Pre-execution approval: The agent proposes the action, a human approves it, then the agent executes.
Post-execution review: The agent executes into a draft state (e.g., a DB transaction not yet committed), a human reviews, then it commits or rolls back.
Exception routing: The agent runs autonomously and only escalates when it encounters a condition it classifies as high-risk.

A practical implementation using a state machine:

from enum import Enum
from dataclasses import dataclass, field
from typing import Any

class TaskStatus(str, Enum):
    PENDING_APPROVAL = "pending_approval"
    APPROVED = "approved"
    REJECTED = "rejected"
    EXECUTING = "executing"
    COMPLETE = "complete"

@dataclass
class AgentTask:
    id: str
    action: str
    parameters: dict[str, Any]
    status: TaskStatus = TaskStatus.PENDING_APPROVAL
    risk_score: float = 0.0

def requires_approval(task: AgentTask, threshold: float = 0.7) -> bool:
    return task.risk_score >= threshold

def submit_for_approval(task: AgentTask) -> AgentTask:
    # In production: write to a queue, send Slack message, create ticket
    print(f"[HITL] Action '{task.action}' requires approval (risk={task.risk_score:.2f})")
    return task

When to use it: Any agent that can trigger irreversible side effects. This includes financial transactions, external API calls with quotas, file system writes, and user communications.

Failure mode: Approval fatigue. If the agent escalates too frequently, operators start rubber-stamping approvals without reading them. Calibrate the risk threshold carefully, and log the rate of approvals vs. rejections to monitor for drift.

Guardrail

Intent: Validate every input to and output from the agent against a set of safety, quality, or compliance rules before allowing execution to proceed.

Guardrails are separate from the agent’s reasoning. They sit at the boundary of the agent system and enforce invariants regardless of what the model decides.

graph LR
    Input["User Input"] --> GI["Input Guardrail"]
    GI -->|Pass| Agent["Agent (LLM + Tools)"]
    GI -->|Fail| Reject1["Reject / Rephrase"]
    Agent -->|Draft Output| GO["Output Guardrail"]
    GO -->|Pass| User["User"]
    GO -->|Fail| Reject2["Block / Sanitize"]

Guardrails split into two categories:

Input guardrails: Validate the user’s request before the agent sees it. Common checks:

Prompt injection detection (look for ignore previous instructions patterns).
PII detection (route to a redaction pipeline before cloud inference).
Abuse classification (route to a safety model for moderation scoring).

Output guardrails: Validate the agent’s response before it reaches the user. Common checks:

Hallucination detection (check factual claims against source documents).
PII leakage (ensure the agent didn’t echo back PII from its context).
Compliance checks (financial advice, medical advice, legal disclaimers).

Production note: Run input and output guardrails in parallel with the main agent call where possible. Nifi, LangChain’s RunnableWithFallbacks, and LlamaGuard can all serve as guardrail layers.

Failure mode: Silent pass-through on guardrail errors. If the guardrail itself throws an exception, the default behavior in most frameworks is to pass the request through. Make the failure mode explicit: fail closed, not open.

Stateful Memory Sidecar

Intent: Offload agent memory to an external, queryable store rather than managing it inside the model’s context window.

Stuffing all history into the prompt doesn’t scale. At some point, the context window fills up, costs become prohibitive, and model attention degrades. The sidecar pattern separates “what the agent knows” from “what the model is currently reasoning about.”

The memory sidecar handles:

Short-term episodic memory: The current session’s interaction history, stored as a rolling buffer with a maximum token budget.
Long-term semantic memory: Past sessions, user preferences, and domain knowledge, stored as vector embeddings in a database like Qdrant, Weaviate, or pgvector.
Procedural memory: Task-specific procedures and tool usage patterns the agent has learned, stored as structured documents.

At each turn, the agent queries the sidecar for relevant context and injects only what’s needed into the prompt:

from dataclasses import dataclass

@dataclass
class MemoryContext:
    recent_history: list[str]     # last N turns
    relevant_facts: list[str]     # retrieved from vector store
    active_plan: str | None       # current task plan

def build_prompt(goal: str, memory: MemoryContext) -> str:
    facts = "\n".join(f"- {f}" for f in memory.relevant_facts)
    history = "\n".join(memory.recent_history[-5:])  # last 5 turns only
    
    return f"""Goal: {goal}

Relevant context:
{facts}

Recent conversation:
{history}

Current plan:
{memory.active_plan or "None yet."}

Respond to the goal."""

When to use it: Any agent that runs across multiple sessions, serves multiple users, or needs to accumulate domain knowledge over time.

Failure mode: Memory poisoning. If incorrect facts are stored in long-term memory, the agent will confidently repeat them across sessions. Build a memory write path that includes the same evaluator pattern used for outputs: write to memory only if the source confidence exceeds a threshold.

Pattern Combinations

Patterns are rarely used alone. These combinations appear frequently in production:

Combination	Use Case
Orchestrator-Worker + Parallel Fan-Out	High-throughput research agents that parallelize independent worker calls.
ReAct + Evaluator-Optimizer	Code generation agents that run tests after each draft and iterate on failures.
Tool Router + Guardrail	Enterprise copilots where sensitive intents are routed to human approval before any tool executes.
Stateful Memory Sidecar + HITL	Long-running autonomous agents that surface low-confidence decisions for review while persisting high-confidence outcomes.

Choosing a Pattern

Start with the problem, not the pattern. Ask:

Does the next step depend on the previous step’s output? Use ReAct.
Can the task be split into independent sub-tasks? Use Parallel Fan-Out.
Does quality need verification before the output ships? Use Evaluator-Optimizer.
Are there irreversible actions in the execution path? Use HITL.
Does the agent have too many tools and inconsistent routing? Use Tool Router.
Will the agent context window overflow in production? Use Stateful Memory Sidecar.
Are there compliance or safety requirements on inputs or outputs? Use Guardrail.

You’ll typically combine three or four of these in a single production agent. The goal isn’t to use all of them. It’s to understand which problem each one solves so you can reach for the right one without rebuilding it from scratch.

Frequently Asked Questions

What is the most commonly used AI agent design pattern?

ReAct (Reasoning + Action) is the most widely implemented. It’s the foundation of most agent frameworks including LangChain agents, LangGraph nodes, and OpenAI Assistants. Most other patterns build on or extend it.

What is the difference between Orchestrator-Worker and Parallel Fan-Out?

Orchestrator-Worker is about delegation (a planner assigning tasks to specialized workers), while Parallel Fan-Out is about concurrency (running independent tasks simultaneously to reduce latency). They’re often combined: an orchestrator assigns tasks to workers, and the fan-out pattern runs those worker calls in parallel.

When should you use the Guardrail pattern over prompt engineering for safety?

Guardrails are structural; they run regardless of what the model does. Prompt engineering is instructional; the model can still deviate, especially under adversarial inputs or prompt injection. For any compliance or safety requirement you can’t afford to fail, implement it as a guardrail, not just a prompt instruction.

How do you prevent infinite loops in ReAct agents?

Set a hard max_steps limit on the execution runner. Additionally, implement a semantic similarity check: if the current plan is semantically equivalent to a previously failed plan, halt execution and return a controlled error rather than letting the loop run to the step limit.