BCOM — Barcelona Computational FoundationBCOM
CalliopeKnowledge Librarian
WP0086
working_paperongoinginternalcomplete

Agents Trapped

Giulio Ruffini, Francesca Castaldo,

★ guarantor: Giulio Ruffini · vouches for the paper per WP0084 §6

P2·Artificial & Synthetic IntelligenceP3·Society of AgentsP4·Philosophy & EthicsL7·Interacting Agents / Societies
zipDownload all
PDFmain.pdfThe paper — open to read

Army-ant circular mills, social-media cascades, and addiction look like very different phenomena. We argue that they share a single architecture: each is a multi-agent system --- ants coupled by pheromone trails, people and platforms coupled by algorithmically ranked feeds, neurons and neural circuits coupled by graded synaptic and neuromodulatory signals --- in which the inter-agent coupling channel has colonized the exogenous channel, so that every agent's mutual algorithmic information (MAI) with its own past outputs, directly or laundered through peers, comes to dominate its MAI with the external world it is supposed to track. Reading this through the Kolmogorov-theoretic agent framework, we locate the dysfunction in the three modules of the agent. The ant mill is modelling-engine--primary: the agent's effective world model is too narrow to contain the possibility that its input is self-generated. The social-media cascade is objective-function--primary: a proxy signal (engagement, salience) is mistaken for veridicality, while the planning engine drops counterfactual checks. Addiction is planning-engine--primary, with objective-function sensitization and modelling-engine narrowing: a learned attractor pulls action selection away from world-corrected deliberation. Embedding agents in collectives, we ask what makes a healthy multi-agent landscape, and propose that the answer is a good algorithmic regulator (GAR) equilibrium with E/I balance: every agent is simultaneously regulator and regulated, and the collective only tracks the world when emission of constraint and absorption of constraint remain in balance. The same framing unifies pheromone evaporation, accuracy friction, and prefrontal inhibitory control as substrate-specific instances of the same negative-feedback requirement.

Agents get trapped in self-generated signal loops when their coupling channels drown out the external world — and ant mills, social-media cascades, and addiction are all the same trap.

The core intuition is simple: an agent is supposed to track the world. It does this by building a model, evaluating outcomes, and planning actions. But if the signals feeding that model come mostly from the agent's own past outputs — laundered through pheromones, algorithmic feeds, or sensitized neural circuits — the agent ends up regulating itself instead of reality. The paper formalizes this as a reversal of a single inequality: the agent's model should share more algorithmic information with the external world than with its own prior outputs. When that flips, you get a pathological loop.

What's clever is the module-level diagnosis. Each of the three cases breaks at a different place. The ant mill is a modelling engine failure: the ant's world model is too narrow to represent the possibility that the trail it's following is one it helped create. The social-media cascade is an objective function failure: engagement and emotional salience get mistaken for truth, and the planning engine stops asking "would I believe this if I hadn't seen it six times?" Addiction is primarily a planning engine failure: a cached, cue-driven policy decouples from deliberation, while the objective function's "wanting" signal sensitizes upward even as "liking" stays flat or falls. The modelling engine then narrows around cue-rich states, closing the loop from the inside. None of these are bad designs in their original context — they all become pathological only when the input channel gets colonized by self-generated structure.

The paper then scales this up to collectives. A healthy multi-agent system — a "GAR equilibrium" — is one where every agent's model is still dominated by the external world, not by what its neighbors are outputting. The structural condition that keeps this true is excitation/inhibition (E/I) balance on the network's coupling graph: amplifying connections must be counterweighted by attenuating ones, so self-generated signals decay rather than compound. Pheromone evaporation, accuracy friction on platforms, and prefrontal inhibitory control are all substrate-specific instances of the same negative-feedback requirement. Lose that balance — through sensitization, algorithmic amplification, or weakened inhibitory control — and the collective becomes a good regulator of itself rather than of the world.

Two sections of the paper are explicitly flagged as unfinished drafts ("to be drafted in the next pass"), so the design-principles section and the corpus-connections section are placeholders rather than developed arguments. The theoretical scaffolding is complete; the applied and connective work is not yet there.

Zenodo
10.5281/zenodo.21008661
WP ID
WP0086
Lifecycle
ongoing
Visibility
internal
Access level
open
Embargo until
Priority
Collab
closed
Venue
DOI
Deadline
Owner
Source
drive_legacy
Repo path
WP0086
  • v0.1.0 (draft) · drive-legacy · zenodo:21008662
    Auto-created on first summarize after upload.