Safety and Failure Modes

"One of us went silent over the river. The rest of us closed the gap, voted down the liar still shouting nonsense, and finished the survey. Nobody on the ground noticed a thing."

A Swarm, Deciding What to Do When One of Its Own Goes Silent

The earlier sections of this chapter built a swarm that works when everything works: agents that perceive, the gossip and consensus that let them agree (Section 39.4), the decentralized controllers that turn agreement into motion (Section 39.6). This section asks the harder question that decides whether such a system is ever allowed to fly: what happens when an agent dies, a sensor lies, a link jams, or a robot is taken over. In a single-machine robot the answer is simple, because the robot either works or it does not, and a watchdog can power it down. A swarm has no such luxury. It is, at any interesting scale, a system in which something is probably already broken, exactly the condition that Section 1.1 named as the price of distribution, now with rotors attached. Keeping it safe means designing so that the broken parts cannot take the healthy ones down with them.

1. The Physical Failure Modes Beginner

A data-center fault is a crashed process or a slow link. An embodied fault is all of that plus physics. It is worth enumerating the failure modes precisely, because each one calls for a different layer of the defense, and conflating them is how swarms hurt people. Four modes dominate. Agent loss is a robot that stops participating entirely: a depleted battery, a structural failure, a hard crash of its onboard computer. To its neighbors this looks exactly like the crashed worker of Section 35.2, detected by missed heartbeats, except that the lost unit is now an unpowered mass on a ballistic path. Sensor failure is subtler and more dangerous: the robot keeps flying and keeps talking, but its perception is wrong, so it broadcasts confident nonsense. Motor or actuator failure degrades the robot's ability to execute the maneuver it agreed to, so its commanded and actual trajectories diverge. Collisions are the failure mode unique to the embodied setting, where two healthy robots, or a robot and the world, occupy the same space, and they are both a cause of the other faults and a consequence of them.

These modes are not independent, and the dangerous cases are the cascades: a sensor failure causes a robot to broadcast a wrong position, its neighbors plan around the phantom, a real gap opens, and a collision closes it. The safety architecture therefore cannot treat any single mode in isolation; it must degrade gracefully under agent loss, reject the bad data from sensor failure, bound the divergence from actuator failure, and hold a collision-avoidance invariant that survives all three. Figure 39.9.1 shows the whole picture: a swarm with one silently lost agent and one Byzantine agent shouting an outlier, the robust aggregate rejecting that outlier, and the formation reconfiguring to cover the gap the lost agent left.

2. Graceful Degradation and Self-Healing Intermediate

When an agent is lost, the swarm must keep doing its job with fewer units, and ideally repair its own structure rather than wait for a human to relaunch a replacement. This is the elastic reconfiguration of Chapter 18 moved from a training cluster to the air. In elastic training, when a worker is preempted the remaining workers re-form the process group and continue with a smaller world size; in a swarm, when a drone drops the remaining drones re-form the formation and continue with smaller coverage. The mechanism is the same membership protocol over missed heartbeats from Section 35.2, and the response is the same: shrink the live set, redistribute the work, carry on.

What changes is the failure metric. A training job cares that progress continues; a survey swarm cares about coverage, the fraction of the target area still observed after units are lost. Suppose $n$ agents are placed so that each covers a region and together they cover the target with some redundancy, so every point is seen by at least $r$ agents. After $k$ agents fail, a point is left uncovered only if all $r$ agents that saw it are among the failed $k$. If failures are spread across the formation, the swarm retains full coverage for any $k < r$ losses, and degrades gracefully (shrinking coverage) rather than catastrophically (sudden blind spots) thereafter. The redundancy factor $r$ is the design knob: a swarm built to survive $k$ simultaneous losses must be flown with

Self-healing closes the loop. Rather than leaving the $n-k$ survivors in their original, now gap-ridden, positions, each survivor runs a local rule that spreads the live agents back out to restore even coverage, a behavior built directly on the decentralized control laws of Section 39.6. The swarm thus heals from the inside: no agent knows the global formation, each only nudges toward its live neighbors, and the formation re-closes the way a torn mesh re-knits. The same membership signal that triggers elastic resize in Chapter 18 here triggers a physical redistribution in space.

3. Byzantine and Compromised Agents in the Swarm Advanced

Agent loss is the easy fault, because a silent agent is at least honest about being gone. The hard fault is the agent that keeps flying and keeps talking but feeds the swarm bad data, whether from a failed sensor or from an attacker who has taken it over. This is the Byzantine fault of Chapter 2, now embodied: a node that can behave arbitrarily, including sending different lies to different neighbors. The classical result from Chapter 2 sets the hard limit. Byzantine consensus among $n$ agents can tolerate $f$ arbitrarily-faulty agents and still let the honest majority agree if and only if

Below that ratio the honest agents can outvote and outweigh any coalition of liars; at or above it the liars can split the honest agents into camps that never converge. For a swarm this is a sizing rule: to survive $f$ compromised units you must fly at least $3f+1$ of them, the embodied cost of Byzantine tolerance.

The agreement that matters in a swarm is usually not a binary commit but a numeric estimate: an obstacle's range, a target's bearing, the formation's centroid. Here the swarm inherits the Byzantine-robust aggregation of Section 35.5 directly. A plain mean of the agents' estimates is unboundedly corruptible, because one agent reporting a wild value drags the mean arbitrarily far. A robust aggregator (the coordinate-wise median, or a trimmed mean that discards the $f$ lowest and $f$ highest reports before averaging) is provably bounded: with at most $f$ liars and $n - f > f$ honest agents whose true values lie in some interval, the median and the trimmed mean stay inside the honest interval no matter what the liars broadcast. Code 39.9.1 demonstrates exactly this contrast on a swarm range estimate.

import numpy as np

# A swarm of n drones each estimates the obstacle's range (meters) from its own
# noisy sensor. They must agree on ONE value to plan a shared avoidance maneuver.
rng = np.random.default_rng(7)
n = 12                                  # swarm size
truth = 30.0                            # true obstacle range in meters
honest = truth + rng.normal(0, 0.4, n)  # honest drones: small sensor noise

# One drone is Byzantine (faulty or spoofed) and broadcasts a wild value that
# would pull the planner into a wrong (here, far too close) maneuver.
byz_idx = 4
reports = honest.copy()
reports[byz_idx] = 1000.0               # the compromised broadcast

f = 1                                   # number of faulty agents
print(f"swarm size n         : {n}")
print(f"faulty agents f      : {f}   (need f < n/3 = {n/3:.2f} for robust consensus)")
print(f"Byzantine report     : {reports[byz_idx]:.1f} m   (truth = {truth:.1f} m)")

# Plain (non-robust) aggregation: the arithmetic mean.
plain = float(np.mean(reports))

# Robust aggregators that tolerate up to f outliers.
median = float(np.median(reports))

# Trimmed mean: drop the f lowest and f highest before averaging (35.5).
srt = np.sort(reports)
trimmed = float(np.mean(srt[f:n - f]))

print()
print(f"plain mean           : {plain:8.2f} m   (corrupted)")
print(f"robust median        : {median:8.2f} m")
print(f"trimmed mean (f={f})   : {trimmed:8.2f} m")
print()
print(f"plain-mean error     : {abs(plain - truth):8.2f} m")
print(f"median error         : {abs(median - truth):8.2f} m")
print(f"trimmed-mean error   : {abs(trimmed - truth):8.2f} m")

swarm size n         : 12
faulty agents f      : 1   (need f < n/3 = 4.00 for robust consensus)
Byzantine report     : 1000.0 m   (truth = 30.0 m)

plain mean           :   110.81 m   (corrupted)
robust median        :    30.01 m
trimmed mean (f=1)   :    30.01 m

plain-mean error     :    80.81 m
median error         :     0.01 m
trimmed-mean error   :     0.01 m

The lesson is the one Figure 39.9.1 drew: the swarm survives a lying member not by detecting and ejecting it (which an attacker can make hard) but by aggregating in a way that the lie cannot move, as long as the liars stay below the $n/3$ fraction. This is fault tolerance by construction, the embodied descendant of the Byzantine-robust gradient aggregation that protected federated training in Section 35.5.

4. Safety Guarantees Under Decentralized Control Advanced

Robust consensus decides what the swarm believes; safety invariants decide what it is allowed to do regardless of that belief. The control-safety invariants of Section 39.6 become the last line of defense precisely because they hold even when consensus is wrong or unavailable. Three invariants matter for an embodied swarm, and all three are enforced locally. Geofencing caps each agent's allowed region: the agent refuses any commanded velocity that would carry it outside a polygon it stores onboard, so no consensus result, honest or malicious, can drive it into restricted airspace. Pairwise collision avoidance guarantees a minimum separation between any two agents from a reciprocal rule each runs against its neighbors, the embodied invariant of Section 39.6, so two robots cannot occupy the same space even if both planned to. Return-to-home is the swarm-wide watchdog: when an agent loses consensus contact, detects critical battery, or trips its own fault check, it abandons the mission and flies a safe recovery path, the embodied analogue of a process that fails fast rather than continuing in an unknown state.

The defining property of these invariants is that they are decentralized and adversary-independent. Geofencing and collision avoidance are checked by each agent on its own commanded motion using only local state, so they keep holding when an agent is cut off from the swarm, when consensus has been corrupted by liars below the $n/3$ bound, or even when a single agent has been compromised entirely (a compromised agent still cannot pull an honest neighbor through its collision invariant, because the honest neighbor enforces separation reciprocally). Safety thus does not depend on the network being up, the consensus being correct, or any monitor being present. It is layered: robust consensus tries to make the swarm believe true things, and the local invariants guarantee that even when that fails, the swarm cannot do an unsafe thing. This layering, a best-effort agreement under a hard local safety floor, is what lets a swarm be both useful and trustworthy without a central authority.

5. Securing the Swarm's Communication Intermediate

Every guarantee so far assumed that an agent's messages reach its neighbors and that a message claiming to come from agent $i$ really did. An adversary attacks exactly those assumptions, turning the comm network of Section 39.4 into the swarm's softest target. Jamming floods the radio band so that messages are lost, which to the membership protocol looks like mass agent loss; the swarm must therefore treat a comms blackout as a safety event, falling back to the local invariants of Section 4 (each agent holds separation and geofence on its own, and triggers return-to-home if isolation persists) rather than continuing to plan on stale agreements. Spoofing is the network-layer face of the Byzantine fault: an attacker injects messages forged to look like a legitimate agent, or replays old ones, attempting to insert exactly the kind of outlier that Section 3's robust consensus is built to reject. The two defenses compose. Authentication (signed, sequence-numbered messages, the security machinery of Section 35.3) shrinks the attacker's reach to nodes whose keys it actually holds, and robust aggregation absorbs whatever forged or compromised reports get through, as long as their count stays below the $n/3$ fraction.

The scale-out lesson sharpens here. A central-monitor design would let an attacker win by taking out one node, the monitor; a swarm has no such single point, which is its security advantage, but it pays for that advantage by needing every agent to authenticate and robustly aggregate on its own, because there is no trusted referee to do it for them. Security in the swarm is therefore not a perimeter around a controller but a property each agent enforces against its own neighbors, which is precisely why it survives the loss or capture of any minority of the fleet.

from scipy.stats import trim_mean
import numpy as np

reports = np.array([30.1, 29.8, 30.3, 1000.0, 29.9, 30.0])  # one Byzantine value
robust = trim_mean(reports, proportiontocut=0.2)            # drop 20% each tail
geometric_med = np.median(reports)                          # coordinate-wise median