"I shook hands with a thousand strangers and remembered every grip. The ones who squeezed too hard, I stopped offering my hand."
An Agent That Keeps a Ledger
In an open multi-agent system, the agents you must cooperate with belong to other owners, were written by other people, and carry no guarantee of competence or honesty; trust and reputation are the mechanisms by which an agent decides whom to rely on anyway. Every coordination mechanism in this chapter assumed a willing, capable partner: the contract-net manager awards a task expecting the winning bidder to deliver, the negotiation counterpart signs a deal expecting it to be honored, the coalition member contributes expecting a fair split. None of that holds automatically when partners can be slow, unreliable, or actively malicious. This section gives an agent the missing faculty: an expectation, learned from experience and from the community, that a given partner will behave as needed. That faculty is the agent-level form of a concern that has run through the entire book, defending a distributed computation against the bad actors and unreliable nodes inside it.
Across this chapter we built up the machinery of cooperation between autonomous agents: a shared language and protocols (Section 29.4), coordination so that joint action stays coherent (Section 29.5), negotiation and auctions to reconcile conflicting goals (Section 29.6), coalition formation, task allocation, and consensus. Every one of those mechanisms quietly assumed that a partner who agreed to do something would actually do it. In a closed system you build, that assumption is reasonable. In an open system, where agents arrive and depart freely and answer to different owners, it is not. A bidder may win a contract and then deliver nothing. A negotiation counterpart may renege the moment the deal stops favoring it. A rater may lie about a competitor to win business. The agent needs a way to assess partners before committing and to track them afterward, so that good experiences and bad ones change who it works with next time.
Two related notions carry this load. Trust is one agent's subjective expectation that a specific other agent will behave as needed in a specific kind of interaction; it is local, personal, and built mostly from that agent's own history with the partner. Reputation is trust aggregated across a community: a summary of what many agents have experienced with a partner, shared so that an agent facing a stranger can borrow others' history instead of starting blind. Trust answers "what do I think of you?"; reputation answers "what does everyone think of you?" The two combine, and the central engineering problem is that the community channel, exactly because it lets you learn from others, is the channel an adversary will try to poison.
1. Trust from Direct Experience Beginner
The simplest and most reliable signal an agent has about a partner is its own history with that partner. Each interaction either succeeds (the task was delivered, the deal was honored) or fails, which makes the natural model a Bernoulli process: the partner behaves well with some unknown probability, and every interaction is a sample. The Bayesian-conjugate way to track that unknown probability is a Beta distribution, and this is exactly the Beta reputation model that underlies many practical systems. An agent keeps two counters per partner, the number of good outcomes $\alpha$ and bad outcomes $\beta$ (each started at one so a stranger begins at the noncommittal middle), and after each interaction increments one of them. The trust score is the posterior mean,
$$T = \frac{\alpha}{\alpha + \beta}, \qquad \alpha \leftarrow \alpha + 1 \;\text{on success}, \qquad \beta \leftarrow \beta + 1 \;\text{on failure}.$$This single line carries more than it looks. The score rises toward one as good interactions accumulate and falls toward zero as bad ones do, so a partner that delivers earns standing and a partner that defaults loses it. The starting values $\alpha = \beta = 1$ encode genuine ignorance about a newcomer, a trust of exactly $0.5$, which is the cold-start problem in miniature: with no history, the agent cannot distinguish a future star from a future cheat, and any policy must decide how much to risk on the unknown. The spread of the Beta distribution, narrow after many interactions and wide after few, even tells the agent how confident it should be in the score, so a trust of $0.8$ from two hundred deals can be weighted differently from the same $0.8$ from two deals.
An agent's own record of what a partner actually did is the one input no other party can fabricate. Everything else, the ratings, endorsements, and reputation scores that arrive from the community, is hearsay that some agent has an incentive to slant. The design rule that follows is blunt: weight direct experience heavily and treat indirect reputation as a discounted prior that direct experience overrides. A reputation system that lets unverified third-party ratings outvote what an agent has seen with its own sensors is a system an adversary will capture, which is precisely the failure the demo in this section reproduces and then fixes.
2. Reputation from the Community, and Its Adversaries Intermediate
Direct experience has a fatal gap: the first time an agent meets a partner, it has none. Reputation fills the gap by letting the agent borrow the community's history. In a centralized design, a reputation store (think of the rating system on any marketplace) aggregates everyone's ratings into a public score; in a decentralized design, agents gossip ratings to each other the way nodes gossip state in Chapter 14. Either way, an agent facing a stranger can consult the aggregate, weight it as a prior, and then let its own subsequent experience pull the estimate toward the truth.
The moment reputation becomes valuable, it becomes worth attacking, and the attacks are systematic. Lying raters submit ratings that do not reflect real experience: a competitor bad-mouths a rival, an ally inflates a friend. Collusion organizes the lying, a ring of agents trading positive ratings to lift each other above honest competitors. The sharpest version is the Sybil attack, in which one party fabricates many identities and has them vouch for each other, manufacturing the appearance of a broad consensus from a single adversary; cheap or free identities make this devastating, because reputation that counts identities rather than verified experience can be inflated without bound. The structural defense is the rule from the previous insight, discount unverified hearsay far below first-hand experience, so that no volume of fabricated ratings can outweigh the defaults an honest agent observes directly.
This is the book's fault-tolerance arc surfacing one final time, now among autonomous agents. In Chapter 2 a Byzantine node was a machine that could send arbitrary, even adversarial, messages, and the defense was an aggregation rule no minority of liars could swing. Trust and reputation are the same idea wearing a different hat: a malicious agent is a Byzantine participant in a social computation, lying raters and Sybils are its arbitrary messages, and discounting hearsay relative to verified experience is the robust aggregation rule. The Byzantine-robust gradient aggregation that defends distributed training against poisoned updates (developed in Chapter 35) and a Sybil-resistant reputation system are answers to one question asked at two scales: how does a distributed computation stay correct when some of its participants are adversarial?
3. A Reputation System That Isolates Bad Actors Intermediate
The claim worth testing is concrete: in a population salted with unreliable and malicious agents, choosing partners by reputation should beat choosing them at random, and it should drive the bad actors out of the market even when those actors actively lie to prop each other up. The simulation below builds twelve agents. Eight are reliable, two are simply unreliable, and the last two are a Sybil pair, one owner running two identities that default on almost every deal yet inject fabricated mutual praise into every other agent's reputation table. Each requester keeps its own Beta reputation over the others, updates it from first-hand outcomes at full weight, and folds in the colluding hearsay at a heavy discount. We run the market under random selection and under reputation-based selection and compare the success rate and how often the Sybils win work.
import random
random.seed(7)
N_AGENTS, N_ROUNDS = 12, 400
# Reliability = chance a chosen partner honors the contract.
reliability = {i: 0.92 for i in range(8)} # honest, reliable
reliability.update({8: 0.20, 9: 0.15, 10: 0.05, 11: 0.05}) # unreliable + Sybils
SYBIL = {10, 11} # one owner, two identities
HEARSAY_WEIGHT = 0.05 # discount on unverified ratings
def fresh_beta(): # Beta(alpha, beta) per partner
return {j: [1.0, 1.0] for j in range(N_AGENTS)}
def rep_score(ab):
a, b = ab
return a / (a + b) # posterior-mean trust
def run(strategy):
rep = {i: fresh_beta() for i in range(N_AGENTS)}
good = bad = sybil_awarded = 0
for _ in range(N_ROUNDS):
for req in range(N_AGENTS):
cands = [j for j in range(N_AGENTS) if j != req]
if strategy == "random":
chosen = random.choice(cands)
else: # pick highest trust, random tie-break
chosen = max(cands, key=lambda j: (rep_score(rep[req][j]), random.random()))
ok = random.random() < reliability[chosen]
rep[req][chosen][0 if ok else 1] += 1.0 # full-weight direct experience
good, bad = good + ok, bad + (not ok)
sybil_awarded += chosen in SYBIL
for req in range(N_AGENTS): # Sybils inject discounted lies
if req in SYBIL:
continue
for s in SYBIL:
rep[req][s][0] += 2.0 * HEARSAY_WEIGHT
return good / (good + bad), sybil_awarded
r_succ, r_syb = run("random")
p_succ, p_syb = run("reputation")
print(f"random selection success : {r_succ:6.1%} sybil awards: {r_syb}")
print(f"reputation selection success : {p_succ:6.1%} sybil awards: {p_syb}")
print(f"success lift : {(p_succ - r_succ):+.1%}")
print(f"sybil awards avoided : {r_syb - p_syb} ({1 - p_syb/r_syb:.0%} fewer)")
random selection success : 65.4% sybil awards: 778
reputation selection success : 91.0% sybil awards: 58
success lift : +25.6%
sybil awards avoided : 720 (93% fewer)The numbers tell the whole story. Random selection treats a reliable agent and a Sybil identically, so it loses a third of its deals and feeds the Sybils a steady stream of work. Reputation-based selection learns, within a few rounds, which identities default and routes around them; the success rate climbs to the reliable agents' own delivery rate, and the Sybils starve. The fabricated praise the Sybils inject is real and present in every reputation table, yet because it is discounted to a twentieth of a first-hand observation, the defaults the honest agents witness directly overwhelm it. Had we instead let the hearsay count at full weight, the Sybils' manufactured consensus would have buried the honest signal and reputation would have performed worse than random, the exact capture the Key Insight warned about.
The Sybil attack is named for a 1973 case study of a single person presenting as sixteen distinct personalities. The computing version is older than the name suggests in spirit: medieval guilds, restaurant guidebooks, and online marketplaces have all fought the same battle against vendors who invent their own glowing reviews. The defense has barely changed in centuries either, trust what you have verified yourself far more than what a stranger insists is true.
4. Trust for LLM Agents That Transact on Our Behalf Advanced
The classic motivation for agent trust was research on software agents trading in electronic markets. That motivation is no longer hypothetical. Large-language-model agents now book travel, negotiate purchases, call external tools, and increasingly transact with other agents in emerging agent marketplaces, all on a human principal's behalf and often with access to that principal's money or credentials. An agent that hires a sub-agent, queries an untrusted tool, or accepts a quote from a counterpart it has never met faces the open-system problem in its sharpest modern form, and the stakes are real funds and real commitments rather than simulated utility.
What makes the modern case harder than the classical one is that the failure mode is no longer just unreliability; it is deception through the very channel the agents communicate on. A malicious party can attempt prompt injection, planting instructions in a web page, a tool result, or a message so that a counterpart agent is hijacked into acting against its principal, a threat we take up in the orchestration setting of Chapter 32. A reputation score does not detect a cleverly injected instruction in a single message, which means trust for LLM agents must be layered: durable identity so a misbehaving agent cannot simply reset its reputation by reappearing under a new name, reputation to track behavior over time, and message-level defenses against injection and deception that operate within each interaction. The faculty this section builds is necessary for that stack but not sufficient on its own.
As LLM agents began transacting autonomously, trust and identity moved from a niche multi-agent topic to active infrastructure work. A line of 2024 to 2025 research and standards activity targets verifiable agent identity and credentials, so that an agent can prove who operates it and carry signed attestations of past behavior, which is the structural counter to Sybil attacks because it makes fresh identities costly. Proposals for agent payment and trust protocols attach reputation and escrow to inter-agent transactions, and benchmarks for agent honesty and deception (work studying when LLM agents lie, sandbag, or are manipulated through their context) are giving the field measurable targets. The open question that ties them together is whether reputation built for slow-moving human markets transfers to agents that can conduct thousands of interactions per minute and can be hijacked mid-conversation by injected text; the classical Beta-reputation core of Code 29.10.1 survives, but it now sits inside a much larger identity-and-safety problem.
The hand-rolled counters in Code 29.10.1 are the whole of a Beta reputation model, so a reusable tracker is genuinely tiny; the value of a real implementation is in the policy around it (decay of stale evidence, confidence from the distribution's spread, robust aggregation of others' reports), not in the update. SciPy's scipy.stats.beta gives the posterior mean and the credible interval for free, collapsing the bookkeeping to a handful of lines:
from scipy.stats import beta
class Reputation: # one tracker per partner
def __init__(self):
self.a, self.b = 1.0, 1.0 # uniform prior: a stranger sits at 0.5
def update(self, success, weight=1.0): # weight < 1 discounts hearsay
self.a += weight * bool(success)
self.b += weight * (not success)
def trust(self):
return beta.mean(self.a, self.b) # posterior-mean trust
def confidence_interval(self):
return beta.interval(0.9, self.a, self.b) # how sure are we?
r = Reputation()
for outcome in [True, True, False, True, True, True]: # six first-hand deals
r.update(outcome)
print(round(r.trust(), 3), tuple(round(x, 3) for x in r.confidence_interval()))
# 0.75 (0.479, 0.947)
scipy.stats.beta supplies both the trust score and a credible interval. The weight argument is the hearsay discount that makes the model Sybil-resistant; production reputation services (and marketplace rating systems) add identity verification and time decay on top of this core.Who: A platform team running an autonomous procurement agent that solicits quotes from supplier agents in an open marketplace.
Situation: The procurement agent awarded purchase contracts to whichever supplier agent showed the highest marketplace reputation, which seemed prudent.
Problem: A low-quality supplier spun up a cluster of shell supplier identities that rated each other five stars, vaulting the ring to the top of the reputation board within a day.
Dilemma: Trust the marketplace's aggregate reputation, which was now corrupted by the Sybil ring, or ignore community reputation entirely and rely only on the agent's own sparse first-hand history, which left it blind to genuine newcomers.
Decision: They kept community reputation but reweighted it, treating aggregate scores as a weak prior discounted to a small fraction of a first-hand delivery and letting the agent's own confirmed outcomes dominate, exactly the HEARSAY_WEIGHT lever in Code 29.10.1.
How: Each supplier started from a discounted community prior; every completed order updated a private Beta reputation at full weight, and suppliers with no verified delivery history were capped at a probationary trust until they earned first-hand evidence.
Result: Within two weeks the Sybil ring's contract share collapsed as the procurement agent's own records showed the shells defaulting, while honest newcomers still climbed once they delivered, reproducing the 93% drop in bad-actor awards seen in Output 29.10.1.
Lesson: Community reputation is a useful prior and a dangerous authority; the robust design lets first-hand experience override the crowd, so no manufactured consensus can outvote what the agent has verified itself.
5. Chapter Summary Beginner
This section completes the chapter's tour of what it takes for autonomous agents to act together. We began with the individual agent that perceives, decides, and acts, and with architectures ranging from fast reactive rules to deliberative planning. We gave agents a shared language and interaction protocols so they could communicate, then coordination so their joint behavior stayed coherent, negotiation and auctions so they could reconcile conflicting goals, coalition formation so they could pool capability, task allocation so work reached the agent best suited to it, and consensus so a group could agree on a single value despite faults. Trust and reputation closed the loop by letting an agent decide whom to rely on at all when the others might be unreliable or adversarial. The throughline is the one the whole book has followed: the same concerns that govern any distributed system, communication cost, coordination, fault tolerance, and robustness against bad actors, reappear here at the level of autonomous decision-making agents, and the solutions rhyme with the ones from earlier parts.
A multi-agent system is a population of autonomous agents that each perceive, decide, and act, built on architectures from reactive to deliberative, communicating through shared protocols, and cooperating through coordination, negotiation, coalition formation, task allocation, and consensus. Trust and reputation let those agents cooperate safely in an open world by learning, from first-hand experience and discounted community evidence, whom to rely on, and by isolating unreliable and malicious actors the way Byzantine-robust methods isolate bad nodes elsewhere in the book. The recurring lesson is that distributing intelligence does not escape the distributed-systems concerns of the earlier parts; it inherits them, so communication, coordination, fault tolerance, and robustness against adversaries return one level up, now between agents rather than between machines.
From here the story splits along two threads the next chapters pick up. When agents must learn their policies by trial and error while every other agent is learning too, coordination and trust become moving targets, and the machinery of distributed reinforcement learning from Chapter 20 must be lifted into the multi-agent setting; that is Chapter 30, multi-agent reinforcement learning. When the agents are LLM-driven and must be wired into reliable, secure pipelines that orchestrate tools, memory, and each other at production scale, the trust and prompt-injection concerns raised in Section 4 become first-class engineering problems; that is Chapter 32, distributed agent orchestration. The game-theoretic foundations from Chapter 28 underpin both.
Two partners both show a trust score of $T = 0.8$. Partner A earned it from $\alpha = 4$ successes and $\beta = 1$ failure; partner B from $\alpha = 80$ and $\beta = 20$. Using the Beta model of Section 1, explain why an agent should treat these two scores differently even though their means are equal, and describe what quantity from the Beta distribution captures that difference. Then state the cold-start problem in your own words and give one concrete policy (other than pure random exploration) by which an agent could responsibly give a brand-new partner a chance to build a record.
Starting from Code 29.10.1, sweep HEARSAY_WEIGHT across the values $\{0.0, 0.05, 0.25, 1.0, 4.0\}$ and plot, for each, the reputation-selection success rate and the number of Sybil awards. Confirm that at full or amplified weight ($1.0$ and $4.0$) the colluding Sybils capture the market and reputation can perform no better than, or worse than, random selection, while small weights isolate them. Then add a third strategy that ignores community ratings entirely (direct experience only) and explain the trade-off it makes against genuine newcomers who have no first-hand record yet.
Extend the model of Code 29.10.1 so that a malicious owner can mint $m$ Sybil identities instead of two, each defaulting and praising the others. Holding the honest population fixed, estimate analytically how the manufactured reputation mass scales with $m$ and the hearsay weight $w$, and derive the condition on $w$ under which first-hand experience still dominates regardless of $m$. Use this to argue why verifiable, costly-to-create identity (the Research Frontier's agent-credentials line) is the structural fix that a discount alone cannot fully provide, and connect your reasoning to Byzantine-robust aggregation in Chapter 35.
These build a working multi-agent system end to end, exercising the chapter's coordination, allocation, and trust machinery together.
1. A reputation system robust to lying raters. Implement a marketplace of agents with a configurable mix of honest raters, biased liars, and Sybil rings, and compare reputation-aggregation rules: naive averaging, hearsay-discounted Beta reputation (this section), and a robust rule that down-weights raters whose reports diverge from an agent's own first-hand outcomes. Measure deal-success rate, time to isolate bad actors, and how each rule degrades as the fraction of liars grows toward and past one half, the Byzantine threshold from Chapter 2.
2. An auction-based task-allocation MAS with reputation-gated bidding. Build a contract-net or single-item-auction market (Section 29.8) in which a manager awards tasks to bidders, but bidders may underbid and then fail to deliver. Add a reputation layer so the manager weights each bid by the bidder's delivered-trust score, and show that reputation-gated awarding raises realized utility over price-only awarding when unreliable bidders are present. Stress-test it against a bidder that builds reputation on small tasks and then defects on a large one.
3. A trust layer for LLM tool-using agents. Wrap a set of LLM agents that call external tools and one another, and attach a Beta-reputation tracker (Code 29.10.2) to each tool and sub-agent keyed to a stable identity. Have the orchestrator route work toward high-trust components and quarantine ones whose outputs repeatedly fail validation, then attempt a prompt-injection attack through a tool result and report whether reputation alone catches it or whether message-level defenses are also required, connecting your findings to Chapter 32.