LLM Agents as Distributed Components

"They call me an agent now, which is generous. I am a process that holds some context, waits on a model I do not own, and pokes the world through tools. Put three of me in a room and you have a distributed system, not a conversation."

An Agent Stuck Waiting on a Lock

The classical agent of Part VI was an abstraction: an entity that perceives its environment, decides what to do, and acts, possibly alongside other agents that it must coordinate with. Chapter 27 framed distributed artificial intelligence around this idea, and Section 29.1 made the perceive-decide-act cycle the unit of a multi-agent system. For most of that history the "decide" step was hand-written: a rule base, a planner, a learned policy. What changed, and what this chapter is about, is that the decide step is now a call to a large language model. That single substitution turns the agent from a self-contained program into a distributed component, because the reasoning no longer happens inside the agent at all. It happens on the distributed LLM-serving fleet of Chapter 24, across the network, on machines the agent does not own and cannot see.

This is the through-line of the chapter. Tool use and function calling (Section 32.2) give the agent its actuators. Roles, planning, and the planner-executor split (Section 32.3) decompose the work. Parallel and distributed workflows (Section 32.4) place agents across machines. Multi-agent reasoning patterns such as debate and reflection (Section 32.5) make several agents argue toward a better answer. Communication protocols (Section 32.6) carry their messages. Shared and distributed memory (Section 32.7) holds their common state. Orchestration engines (Section 32.8) run the whole graph. Evaluation (Section 32.9) measures whether it works, and operating at scale (Section 32.10) confronts the cost, latency, and reliability that distribution imposes. Every one of these is a distributed-systems topic wearing an agent costume. The figure below shows the costume and the system underneath it.

1. The Agent Is a Stateful Service, Not a Prompt Beginner

A useful first move is to stop thinking of an agent as a block of prompt text and start thinking of it as a service with an address, some state, and a request-response loop. The state is the agent's context: the running transcript of what it has perceived, what it has decided, and what its tools returned. A request arrives (a task, or a message from another agent), the agent assembles its context into a prompt, sends that prompt to the model, receives a decision, and either acts on the world through a tool or emits a message to another agent. Then it waits for the next input. This is precisely the shape of any networked service, and it is precisely the perceive-decide-act cycle of Section 29.1, with the decision delegated to a remote model.

Seeing the agent as a service immediately surfaces the distributed-systems questions. Where does the state live, and what happens to it if the process restarts? How long does the reasoning step block, and what is its tail latency, given that it is a call to a shared fleet with its own queue? What does the agent do when the model call times out or returns malformed output? These are not prompting questions; they are the questions Chapter 2 asked about any distributed component, and they have the same answers here. The agent is a component, the reasoning is a remote procedure call, and the rest follows.

2. A Single Agent, Built From Scratch Beginner

To make the component concrete before we connect many of them, we build the smallest honest agent: a perceive-reason-act loop that calls a model and one tool to finish a small task. The reasoning step is a stub that returns canned decisions so the loop runs offline and deterministically; mentally, that stub stands in for the network call to the serving fleet, and nothing in the loop changes when you swap the stub for a real client. The tool is a pure calculator. The task is to compute an order total. Watching the loop turn perceptions into actions is the whole point: it shows that an agent is a control loop around a reasoning call, and the rest of the chapter is about distributing that loop.

"""A single LLM agent as a perceive-reason-act loop. No network, no real LLM.
The LLM is a deterministic stub that returns canned decisions, and the one tool
is a pure-Python calculator. This makes the agent-as-component concrete."""

import json


def stub_llm(messages):
    """A stand-in for a call to the distributed LLM-serving fleet (Chapter 24).
    Real systems send `messages` over the network and wait; here we pattern-match
    the latest observation and return a canned decision, so the loop is offline."""
    last = messages[-1]["content"]
    if "TASK:" in last:
        return {"thought": "I must compute the total, so I will call the tool.",
                "action": "calculator", "args": {"expr": "(149.99 * 3) + 24.50"}}
    if "TOOL_RESULT:" in last:
        value = last.split("TOOL_RESULT:")[1].strip()
        return {"thought": "I have the number; I can answer now.",
                "action": "finish", "args": {"answer": f"The order total is {value}."}}
    return {"thought": "Unclear state.", "action": "finish", "args": {"answer": "?"}}


def calculator(expr):
    """The single tool this agent can act with: a pure, side-effect-free function.
    In a real system a tool is often a remote service behind its own network call."""
    return str(round(eval(expr, {"__builtins__": {}}, {}), 2))


TOOLS = {"calculator": calculator}


def run_agent(task, max_steps=4):
    """The perceive-reason-act loop of Chapter 29, made into a service component."""
    messages = [{"role": "user", "content": f"TASK: {task}"}]
    for step in range(1, max_steps + 1):
        decision = stub_llm(messages)                      # REASON (the LLM call)
        print(f"step {step}: thought  -> {decision['thought']}")
        action, args = decision["action"], decision["args"]
        if action == "finish":                             # ACT (terminal)
            print(f"step {step}: finish   -> {args['answer']}")
            return args["answer"]
        result = TOOLS[action](**args)                     # ACT (tool call)
        print(f"step {step}: action   -> {action}({json.dumps(args)}) = {result}")
        messages.append({"role": "assistant", "content": json.dumps(decision)})
        messages.append({"role": "user", "content": f"TOOL_RESULT: {result}"})  # PERCEIVE
    return None


if __name__ == "__main__":
    run_agent("Three items at 149.99 plus 24.50 shipping; what is the order total?")

step 1: thought  -> I must compute the total, so I will call the tool.
step 1: action   -> calculator({"expr": "(149.99 * 3) + 24.50"}) = 474.47
step 2: thought  -> I have the number; I can answer now.
step 2: finish   -> The order total is 474.47.

Three things in Code 32.1.1 are worth dwelling on because they become the chapter's themes. First, the agent never computes anything itself except by acting through a tool; its only native ability is to assemble context and consult the model, which is why tool use (Section 32.2) is the agent's hands. Second, the loop is bounded by max_steps, because a reasoning call can fail to terminate, and a real orchestrator must cap and supervise these loops, a reliability concern (Section 32.10). Third, every iteration appends to messages, so the context grows; in a multi-agent system that growing context is shared state that several agents may read and write, which is the consistency problem of distributed memory (Section 32.7). One small loop already contains the seeds of communication, coordination, consistency, and reliability.

3. Why Many Agents Make a Distributed System Intermediate

Place a second agent beside the first and the four classical concerns of distributed computing arrive together. Consider them in turn, because the chapter is organized around them. Communication is the first and the most unusual. In the systems of earlier parts a message was a packet, costed in microseconds and bytes; here a single message from one agent to another typically triggers a reasoning call, so its cost is a full LLM inference, costed in hundreds of milliseconds and in dollars. Communication, which the whole book has taught you to minimize, is now the dominant expense, and an agent graph with many edges is a graph with many expensive calls. If an agent makes $r$ reasoning calls to complete its part and there are $A$ agents, a naive all-to-all discussion can cost on the order of

where $c_{\text{call}}$ is the cost of one model call. Doubling the number of agents or the number of rounds does not split work the way data parallelism did in Section 1.1; it multiplies cost. That asymmetry, more agents meaning more cost rather than less time, is why Section 32.10 treats topology as a budget decision.

Coordination is the second concern. Several agents working on one task must agree on who does what and in what order, which is the coordination problem of Chapter 29 and the collective-behavior problem of Chapter 31, now with reasoning agents as the participants. Consistency is the third: when agents share a scratchpad, a plan, or a memory store, they can read stale versions and act on them, the same staleness that Chapter 2 analyzed for any replicated state. Reliability is the fourth: any agent's reasoning call can time out or return garbage, any tool can fail, and the orchestrator must detect, retry, and contain these failures, exactly the discipline of Chapter 35. None of these four is new. What is new is that the participants are LLM agents and the messages are reasoning calls, which changes the constants but not the questions.

4. Orchestration Is a Distributed-Systems Discipline Intermediate

The central claim of this chapter, and the reason it sits in a distributed-systems book rather than a prompting guide, is that the quality of a multi-agent system is governed by its system design and not by the wording of its prompts. Two teams can give their agents identical prompts and reach opposite outcomes because one designed the communication topology, the state-sharing scheme, and the failure handling deliberately and the other left them implicit. A star topology with a single coordinator has different cost, latency, and failure characteristics than a fully connected debate; a shared blackboard has different consistency guarantees than per-agent private memory; a pipeline of specialists has different tail latency than a parallel ensemble. These are architecture choices, and they are the choices this book has spent thirty-one chapters teaching you to make.

This reframe pays off immediately in practice. When a multi-agent system is slow, the fix is usually not a better prompt but fewer reasoning rounds or more parallelism across agents. When it is expensive, the fix is a sparser communication topology, not a shorter system message. When it gives inconsistent answers, the cause is often two agents acting on divergent copies of shared state, a consistency bug, not a reasoning failure. Diagnosing agent systems with a distributed-systems vocabulary, latency and throughput, topology and consistency, retries and timeouts, is what makes them debuggable. The modern frameworks recognize this explicitly: they model an agent system as a graph of components with typed edges and shared state, which is to say, as a distributed system.

# pip install langgraph
from langgraph.graph import StateGraph, END

g = StateGraph(dict)                       # shared state flows along the edges
g.add_node("reason", call_model)           # the remote LLM step (our stub_llm)
g.add_node("act", run_tool)                # the tool step (our calculator)
g.set_entry_point("reason")
g.add_conditional_edges("reason",          # route on the model's decision
    lambda s: "act" if s["action"] != "finish" else END)
g.add_edge("act", "reason")                # loop back to reason after acting
app = g.compile(checkpointer=saver)        # persistence and recovery for free

We now have the chapter's foundation: the agent as a stateful distributed component whose reasoning is a remote procedure call (Section 1 and 2), the four classical concerns that arrive the moment a second agent appears (Section 3), and the reframe that makes the rest of the chapter coherent, that orchestrating agents is systems engineering (Section 4). The next section gives the agent its hands, examining tool use and function calling as the mechanism by which an agent's reasoning reaches out and acts on the world. That mechanism, and the distributed calls it triggers, begins in Section 32.2.