What Representation Theorems Do---and Do Not---Say About Utility and Agents

Representation theorems are boundary markers, not death sentences — they tell you exactly when a scalar utility works, not whether one can exist at all.

The core confusion this paper clears up is a common one: people see the Allais paradox or Ellsberg's urn experiments, conclude that "utility theory is dead," and move on. That's wrong. What those experiments actually show is that specific axioms — like the independence axiom in von Neumann-Morgenstern, or Savage's sure-thing principle — are too strong to describe human behavior in certain tasks. They don't show that no scalar objective can organize behavior. The paper is essentially a careful map of what each major theorem requires and what it delivers, so you know exactly which tool to reach for and when to switch.

The classic theorems are if-then statements. VNM says: if your preferences over lotteries satisfy completeness, transitivity, continuity, and independence, then there exists a utility function such that you rank lotteries by their expected value under it. Savage says: if your preferences over acts satisfy a similar list (including the sure-thing principle), then you have both a subjective probability and a utility, derived jointly from behavior alone. Debreu says: if preferences over commodity bundles are complete, transitive, and continuous, a real-valued utility exists. Afriat says: if a finite dataset of observed choices satisfies GARP (a consistency condition on revealed preferences), then some well-behaved utility rationalizes it — no functional form assumed. Each theorem earns its conclusion by imposing structure. When the structure fails, the conclusion doesn't follow — but that's a limitation of the form, not a refutation of utility itself.

The paper then connects this to the KT agent framework, where an agent has a scalar "Objective Function" evaluated not on raw outcomes but on its internal world model. This is the key architectural move. Context dependence, reference points, and apparent preference reversals can all emerge from a stable master objective (the paper calls it "telehomeostasis" — roughly, expected long-run fitness) applied to a context-sensitive internal model. The Allais pattern, for instance, can be rational if the agent's model includes a binding financial threshold that only the high-variance lottery clears. Same numbers, different world model, different optimal action — and no contradiction with a single stable objective.

Bounded rationality fits cleanly here too. Real agents have limited memory, attention, and compute. They make mistakes. But mistakes are part of the model, not evidence against it. If an oracle told the agent which action best serves its goals and the agent consistently accepted that advice, the "irrational" choices were resource limitations, not the absence of a scalar objective. The paper cites random utility models, rational inattention, and satisficing as standard ways to formalize this — all of which preserve goal-directedness while accommodating noisy or approximate optimization.

The practical upshot is a decision tree for modelers: match the framework to the environment. Objective lotteries call for VNM. Subjective uncertainty with context-separability calls for Savage. Observational data with minimal structure calls for Afriat. When axioms fail, don't abandon the agent model — extend the outcome space, relax the offending axiom, or add state-dependence. The theorems are diagnostics, not verdicts.