Centralized, Decentralized, and Hybrid AI

"They put me in charge of everyone's plans. For a glorious afternoon I was optimal. Then I crashed, and twelve agents discovered they had never learned to ask each other anything."

A Coordinator That Was a Single Point of Failure

Distributed problem solving, the subject of Section 27.2, asked how a group of agents can jointly solve a problem none of them holds entirely. It left open the question of authority: when the agents must agree on who does what, who decides? That question has the same three answers it had for distributed training, because it is the same question one level up. In data-parallel training the unit of coordination is a gradient and the authority structure decides how partial gradients are combined. In a multi-agent system the unit of coordination is a decision and the authority structure decides how partial intentions are combined. Recognizing that these are the same spectrum, viewed at the data level and at the reasoning level, lets us reuse everything Chapter 2 taught about consensus, bottlenecks, and failure.

1. Centralized: One Agent Plans for Everyone Beginner

In a centralized architecture a single coordinator agent holds, or gathers, the information needed to plan, decides the allocation of work, and instructs the others to carry it out. The workers report what they can do and at what cost; the coordinator solves the resulting optimization and pushes assignments back. This is the structure of the orchestrator pattern that dominates today's large-language-model agent frameworks: one planner model reads the task, decomposes it into subtasks, and dispatches each to a tool or a subordinate executor. Because the coordinator sees everything, it can in principle compute a globally optimal plan, and because the logic lives in one place, the system is simple to reason about, debug, and observe. There is one log, one decision-maker, one place to look when something goes wrong.

The cost of that simplicity is structural. Every decision flows through one agent, so the coordinator's throughput caps the whole system's, exactly the bottleneck that Chapter 2 identified for any single point of coordination, and gathering global state before each decision is itself a communication round that grows with the number of agents. Worse, if the coordinator fails, the system has no plan and no way to make one; the workers were never given the authority or the information to continue. Centralization concentrates both capability and fragility in the same node.

2. Decentralized: Peers Coordinate with Local Information Intermediate

In a decentralized architecture there is no coordinator. Each agent knows only its own state and what its neighbors tell it, and the global allocation emerges from many local interactions: agents announce tasks, bid on the ones they can do cheaply, and award them by a local rule, with no node ever holding the whole picture. This is the world of the Contract-Net protocol that Section 27.5 formalizes, of the swarms in Chapter 31 where simple peers produce complex collective behavior, and of gossip-style coordination where information diffuses through pairwise exchange rather than a central broadcast. Because no single node is essential, the loss of any one agent degrades the system gracefully rather than halting it, and because work is shared by exchanging messages with neighbors rather than funneling through one node, the design scales to far more agents than a single coordinator could serve.

What decentralization gives up is the global view, and with it the easy guarantees. No agent can certify that the emergent allocation is globally optimal, because no agent ever sees the global cost; the system can settle into a locally sensible but globally suboptimal arrangement. Coordination also costs more messages, since agreement that a coordinator could reach by reading shared state must instead be negotiated through rounds of announcements and bids. Observability suffers too: there is no single log of "the plan", only a distributed state scattered across agents that must be reconstructed to be understood. These are the reasoning-level forms of the resilience-for-coherence trade that gossip averaging made at the data level in Section 1.4.

3. Hybrid: Hierarchies of Coordinators Intermediate

Real systems rarely sit at either pole. The common practical middle is hierarchical: local coordinators each manage a small team with the simplicity and near-optimality of centralization within their group, and a global coordinator composes the teams without having to plan for every agent directly. A global planner that had to assign work to a thousand agents one by one would be an impossible bottleneck; a thousand peers negotiating pairwise would drown in messages and never converge to anything coherent. A two-level hierarchy of, say, ten local coordinators each managing a hundred agents keeps each coordinator's problem small while preserving a single point of global coherence. This is exactly the hierarchical agent-team structure that Chapter 32 builds for orchestrating large fleets of language-model agents, and it mirrors the hierarchical-then-flat collective topologies that made all-reduce scale in Chapter 4.

The hybrid design is a tunable dial, not a fixed point. Push authority up and you approach centralization: stronger global guarantees, weaker fault tolerance. Push it down and you approach decentralization: more resilience and scale, weaker global coherence. Choosing where to set the dial is the architectural decision, and it is made per workload by asking which failures you can tolerate and which guarantees you cannot do without.

4. The Same Problem, Two Architectures Intermediate

To make the trade-offs concrete rather than rhetorical, we solve one task-allocation problem two ways and measure what differs. There are $K$ agents and $M$ tasks; assigning task $t$ to agent $a$ costs $c_{a,t}$, and we want an assignment $\pi$ minimizing the total,

The centralized coordinator reads the whole cost matrix and picks, for each task, the agent with the lowest cost, a global argmin per column that is optimal for this unconstrained version. The decentralized peers never see the matrix; for each task they run a Contract-Net round, announcing the task, collecting one bid per agent, and awarding it to the lowest bidder. With no capacity limits the two converge to the identical allocation, which is exactly what lets us isolate the real differences: how many messages each spends, and what happens when an agent fails. The code below runs both and then kills agent 2 to test robustness.

import numpy as np

# A task-allocation problem: M tasks must each be assigned to exactly one of
# K agents. Cost c[a, t] is what agent a pays to do task t. We minimize total
# cost. We solve it two ways and compare quality, messages, and robustness.

rng = np.random.default_rng(7)
K, M = 5, 12                                   # agents, tasks
cost = rng.integers(1, 100, size=(K, M))       # cost[a, t]: agent a doing task t


def total_cost(assign, cost, dead=None):
    # assign[t] = agent index handling task t; dead agents cannot count.
    tot, unassigned = 0, 0
    for t, a in enumerate(assign):
        if a is None or (dead is not None and a == dead):
            unassigned += 1
        else:
            tot += cost[a, t]
    return tot, unassigned


# ---- Centralized: one coordinator sees the full cost matrix and optimizes. ----
def centralized(cost):
    K, M = cost.shape
    assign = [int(np.argmin(cost[:, t])) for t in range(M)]   # global argmin per task
    msgs = K + M                  # K agents report costs in, M assignments sent out
    return assign, msgs


# ---- Decentralized: peers bid via a Contract-Net-style round, local info only. ----
def decentralized(cost, dead=None):
    K, M = cost.shape
    assign = [None] * M
    msgs = 0
    for t in range(M):
        # Announce task t to all agents; each alive agent replies with one bid.
        bids = {}
        for a in range(K):
            if dead is not None and a == dead:
                continue
            msgs += 2                          # announce + bid for this (task, agent)
            bids[a] = cost[a, t]
        if bids:
            winner = min(bids, key=bids.get)   # lowest local bid wins; award message
            msgs += 1
            assign[t] = winner
    return assign, msgs


c_assign, c_msgs = centralized(cost)
c_cost, _ = total_cost(c_assign, cost)
d_assign, d_msgs = decentralized(cost)
d_cost, _ = total_cost(d_assign, cost)

print("agents K, tasks M       :", K, M)
print("centralized total cost  :", c_cost, " messages:", c_msgs)
print("decentralized total cost:", d_cost, " messages:", d_msgs)
print("same allocation         :", c_assign == d_assign)

# ---- Robustness: agent 2 fails AFTER the coordinator already assigned work. ----
dead = 2
c_after, c_un = total_cost(c_assign, cost, dead=dead)
# Decentralized re-runs its bidding round with the failed agent simply absent.
d_assign2, d_msgs2 = decentralized(cost, dead=dead)
d_after, d_un = total_cost(d_assign2, cost, dead=dead)

print("\n-- agent", dead, "fails --")
print("centralized   : tasks dropped =", c_un, " (frozen plan, no re-bid)")
print("decentralized : tasks dropped =", d_un,
      " recovered cost =", d_after, " extra messages:", d_msgs2)

agents K, tasks M       : 5 12
centralized total cost  : 223  messages: 17
decentralized total cost: 223  messages: 132

-- agent 2 fails --
centralized   : tasks dropped = 1  (frozen plan, no re-bid)
decentralized : tasks dropped = 0  recovered cost = 229  extra messages: 108

The numbers tell the whole story of the section. Solution quality was identical because the problem was easy and the global view bought nothing the local bids could not find, which is common: centralization's optimality edge only matters when constraints couple the tasks so that no greedy local rule suffices. The message cost was where the architectures parted, the decentralized protocol paying a large multiple for coordinating through negotiation rather than reading shared state. And robustness inverted the picture: the very central plan that was cheap to compute became a liability the moment its assumptions changed, because no worker had the authority or information to repair it, whereas the decentralized system treated a failed agent as just one fewer bidder. Faster and cheaper to plan, but brittle, versus costlier to coordinate, but self-healing, is the trade you are always making.

5. Observability and the Cost of a Global View Advanced

A theme that the experiment surfaces but does not fully name is observability, the practical cost of knowing what the system is doing. Centralization makes observability nearly free, because the coordinator already holds the global state that a monitor wants to read; one log captures the plan, and one query answers "what is the system doing?" Decentralization scatters that state across agents, so reconstructing a global view becomes its own distributed computation, with the same staleness and consistency caveats that Chapter 2 attached to any global snapshot. This is why decentralized systems are harder not only to guarantee but to debug: the information needed to understand them is exactly the information they were designed not to centralize.

The cost of a global view, then, is paid in one of two currencies. A centralized system pays it up front and continuously, as the communication and bottleneck cost of gathering global state before every decision. A decentralized system avoids that standing cost but pays a lump sum whenever it actually needs the global view, for monitoring, for a global guarantee, or for a human to understand what happened. Hybrid architectures let you choose how much of each currency to spend, which is the deeper reason they dominate practice: most systems need a strong-enough global view often enough that pure decentralization is inconvenient, but not so constantly that pure centralization is affordable.

from langgraph.graph import StateGraph, END

# Centralized: every executor edge returns to the coordinator, which re-plans.
g = StateGraph(State)
g.add_node("coordinator", plan_and_assign)        # one node holds the plan
for worker in ("billing", "refunds", "recovery"):
    g.add_node(worker, run_worker)
    g.add_edge(worker, "coordinator")             # report back to the center
g.add_conditional_edges("coordinator", route_or_finish)

# Decentralized: peers hand off to each other directly, no return-to-center.
g2 = StateGraph(State)
for peer in ("billing", "refunds", "recovery"):
    g2.add_node(peer, run_peer_with_handoff)      # each peer decides the next hop
    g2.add_conditional_edges(peer, pick_next_peer_or_end)

We now have the control-architecture spectrum for intelligence, the matching trade-offs from distributed systems, and a measured sense of what centralization and decentralization actually cost. One specific coordination pattern deserves its own treatment, because it organizes decentralized agents around a shared structure rather than around a coordinator or pure peer messaging: the blackboard, where agents read and write a common workspace and the global solution accretes on it. That architecture is the subject of Section 27.4.