Mathematical Foundations of the Algorithmic Agent (v2)

A rigorous answer to "what separates a thermostat from a rock?" — formalized as three axioms that carve out a proper, non-trivial class of algorithmic agents from the space of all input-output functions.

The core problem is an old one in philosophy of mind: if you define "agent" too loosely, everything qualifies. Putnam showed that any rock can be mapped onto any finite-state automaton if you're allowed to pick your state predicates post-hoc. The same threat applies to agency — define it as "something that maps inputs to outputs" and every lookup table is an agent. This paper, building on WP0018 and the Algorithmic Regulator Theorem (ART), sharpens a formal definition that actually excludes the impostors.

The definition has three jointly necessary conditions. First, the system must maintain a compressive world model — not just a table of observed inputs, but an internal representation that shares mutual algorithmic information with the process generating the data. This is a strong condition: a lookup table can memorize a finite dataset but fails on new samples from the same process. Second, the system must have a non-trivial objective function — something that evaluates model states differently, not a constant. Third, it must perform counterfactual planning — its action selection must depend on what would happen under alternative choices, not just what has happened. Each axiom excludes a distinct class of non-agents: lookup tables fail on the first, pure data loggers on the second, reflex arcs on the third.

The paper then proves two things: the class is non-empty (a bang-bang thermostat satisfies all three axioms, and so does an analog bimetallic-strip thermostat — no software required), and the class is strictly smaller than all I/O functions (constant emitters, identity transducers, lookup tables, and random oracles are all formally excluded). The thermostat-vs-rock distinction is made precise: the strip's curvature reliably covaries with temperature across conditions because of physical law, not because someone cherry-picked predicates to match one trajectory.

The paper also provides a measurable proxy for agency: the compressibility gap Δ from the ART, defined as how much a regulator reduces the algorithmic complexity of the world's output. A large, persistent Δ > 0 across diverse tasks is evidence the system carries genuine model content — it couldn't be faked by a lookup table. Crucially, Δ is estimable in practice using standard compression algorithms (gzip, BDM, neural compressors), bridging the abstract definition to empirical use.

A rich survey maps the {M, O, S} factorization — model, objective, selector — onto systems from bacteriophage λ (a bistable genetic switch as minimal planner) through E. coli chemotaxis to scaffolded large language models. The paper argues that a bare LLM is not an agent (no intrinsic objective, no planning loop) but a scaffolded LLM is, with the weights serving as the model, the system prompt as the objective, and chain-of-thought reasoning as counterfactual planning. The framework also connects to consciousness: the paper argues that any system satisfying the three axioms has the structural prerequisites for what KT calls "structured experience" — model content, valence, and arousal — reframing the binary "is it conscious?" question as a graded one about richness across three dimensions.