Building a Multi-Agent System from Scratch: Architecture Patterns

A single well-tuned agent is easy to demo. Give it a clear goal, a handful of tools, and a generous system prompt, and it'll look impressive in a five-minute walkthrough. I wrote about exactly that kind of agent in an earlier post — a single agent that turned natural-language restaurant queries into structured search filters. It worked well because the problem had one shape: understand the query, call some tools, format the answer.

Multi-agent systems are a different animal. The moment you have more than one agent, you've introduced a coordination problem on top of the reasoning problem, and coordination is where almost all of the real engineering effort goes. Nobody warns you about this in the demos, because demos are built to succeed once, not to run in production for six months against inputs nobody anticipated.

This post is about the three architecture patterns I keep coming back to when splitting work across multiple agents, and — more importantly — the failure modes that only show up once you're past a single-agent demo and into something that has to run unattended.

Pattern 1: Orchestrator-Worker

This is the pattern most people reach for first, and for good reason — it maps cleanly onto how a human manager delegates work. One orchestrator agent receives the task, breaks it into subtasks, dispatches each subtask to a specialized worker agent, and aggregates the results into a final answer. Workers don't talk to each other; they only talk to the orchestrator.

Task → Orchestrator
              ├─→ Worker A (research)
              ├─→ Worker B (code generation)
              └─→ Worker C (validation)
                     ↓
             Orchestrator aggregates → Final answer

This pattern fits when the task decomposition is clear upfront — you know in advance that answering the question requires "look something up," "generate something," and "check it," even if you don't know the specifics until runtime. The orchestrator's only job is planning and aggregation, which keeps its prompt small and its behavior predictable. The tradeoff is that the orchestrator becomes a bottleneck: every piece of context has to flow through it, and if it misjudges the decomposition, every worker downstream inherits that mistake.

Pattern 2: Blackboard

The blackboard pattern drops the strict hierarchy. Instead of a central orchestrator directing traffic, agents read from and write to a shared state store — the "blackboard" — and each agent decides for itself when it has something useful to contribute. There's usually a lightweight scheduler that decides which agent gets to act next based on what's currently on the blackboard, but there's no single agent that owns the plan.

         ┌────────────────────────┐
         │   Shared Blackboard    │
         │  (partial state,      │
         │   hypotheses, facts)  │
         └───────────┬────────────┘
        read/write    │    read/write
     ┌─────────────────┼─────────────────┐
     ↓                 ↓                 ↓
 Agent A           Agent B           Agent C
(hypothesis)      (evidence)        (critique)

I reach for this pattern when the solution path genuinely isn't known ahead of time — debugging a subtle production issue, for instance, where one agent proposes a hypothesis, another gathers evidence for or against it, and a third critiques the reasoning. Nobody could have written that as a fixed sequence of steps beforehand, because the next useful action depends entirely on what's already been discovered.

The cost of that flexibility is that blackboard systems are much harder to reason about and to test. There is no single place where "the plan" lives, so debugging a bad outcome means reconstructing the entire sequence of reads and writes after the fact — closer to debugging a race condition than debugging a function call stack.

Pattern 3: Pipeline

The pipeline pattern is the simplest of the three, and it's the one I default to whenever the workflow is well-understood and repeatable. A fixed sequence of specialized agents, each handing its output to the next: retrieve, extract, validate, format. No delegation logic, no shared scratch space — just a chain.

Input → Retrieve Agent → Extract Agent → Validate Agent → Format Agent → Output

Pipelines are the easiest of the three patterns to evaluate, because each stage has a well-defined input and output contract, which means you can test and version each agent independently. The obvious limitation is rigidity: if a task doesn't decompose into a fixed sequence — if step three sometimes needs to loop back and redo step one — a pipeline will either fail outright or you'll end up bolting ad hoc branching logic onto it until it quietly turns into an orchestrator with extra steps.

Failure Modes You Won't See in a Demo

Every one of these showed up for me only after moving past a controlled demo into something running against real, messy inputs.

Infinite delegation loops

Two agents that can each decide "this isn't my job, pass it back" will occasionally pass a task back and forth indefinitely. An orchestrator delegates to a worker; the worker decides it needs clarification and delegates back to the orchestrator; the orchestrator, seeing an unclear request, delegates it right back out. Neither agent is wrong in isolation — the loop only exists at the system level, which is exactly why it's invisible when you're testing agents individually.

Runaway cost from re-invocation

Every agent-to-agent call is a model call, and model calls cost money and time. A blackboard system with three agents each re-checking the board after every write can spiral into dozens of model calls for a task that a human would resolve in three steps. This is easy to miss in development, where you're running one task at a time and eyeballing the transcript. It's very much not easy to miss on the invoice.

Conflicting concurrent writes

When two agents can act on shared state at the same time — two workers both trying to update the same record, two blackboard agents both trying to "resolve" the same open question — you get the same class of bug as an unguarded race condition in concurrent code, except the two writers are language models instead of threads, which makes it much harder to reproduce reliably.

Error propagation between agents

The most insidious one. If Agent A hallucinates a fact and passes its output to Agent B, Agent B has no way to distinguish that hallucination from a verified fact — it just sees text in its context window and reasons from it as if it were true. In a single-agent system, a hallucination is a wrong answer. In a multi-agent system, a hallucination is an input that gets compounded by every downstream agent that trusts it.

Guardrails That Actually Work

None of the failure modes above are solved by "better prompting." They're solved by hard limits enforced in code, outside the agents' control.

The Honest Summary

Multi-agent architecture buys you task decomposition — the ability to break a large, fuzzy problem into smaller, more tractable pieces that are each easier to prompt, test, and reason about. It does not, on its own, buy you correctness. A system of five agents that each hallucinate 5% of the time doesn't average out to more reliable behavior; it compounds. The pattern you choose determines how the work is split. The evaluation harness and the guardrails you build around it determine whether the system is safe to actually put in front of users. Spend your engineering time accordingly.