Kolmogorov Theory and the Free Energy Principle

Two independently invented theories of mind — one from algorithmic compression, one from Bayesian physics — turn out to be describing the same thing from different altitudes.

Kolmogorov Theory (KT) starts from a simple observation: any system that persists in a changing world must compress that world into a model and act to stay viable. It doesn't care what the system is made of — a cell, a brain, a glider in Conway's Game of Life. It just asks: what must any such bounded agent do? The answer is build compact predictive models, score states by some objective, and plan accordingly. This is a computational-level account — it says what is being computed, not how.

The Free Energy Principle and Active Inference (FEP/AIF) arrive at the same agent from a completely different direction: stochastic dynamics, Bayesian inference, and the physics of self-organization. A living system is separated from its environment by a Markov blanket — a statistical boundary where internal states only "see" the outside world through sensory states, and only "touch" it through active states. Given that structure, the natural thing to minimize is variational free energy: a quantity that upper-bounds surprise and decomposes into prediction error plus the cost of updating beliefs. Goals are encoded as prior preferences inside the generative model, so exploration and exploitation both fall out of one unified objective. FEP/AIF adds what KT cannot supply on its own: concrete stochastic dynamics, a physics of beliefs, and a process theory with plausible neural implementation.

The paper's central argument is that these two traditions are conditionally equivalent. KT drives any bounded agent toward probabilistic inference (because exact Kolmogorov complexity is uncomputable — you can't find the shortest program, so you maintain a weighted portfolio of candidates). Once you additionally commit to reading the agent's information boundary as a Markov blanket and encoding goals as prior preferences, variational free energy becomes the canonical objective. Active inference is then one particularly powerful member of a family of bounded-agent architectures — not the unique destination, but a biologically well-motivated one.

The most interesting architectural difference is in how the two handle goals. FEP/AIF folds valuation into the generative model itself — exploration and exploitation are not separate drives but both emerge from minimizing one expected free energy. KT keeps the objective function as a separate module, which makes it easier to ask whose persistence an agent is serving, and to handle non-stationary or externally imposed goals. The paper argues this isn't just a design preference: the separable form is diagnostically useful for artificial agents, alignment problems, and multi-scale biological systems where a brain might simultaneously serve the individual, the lineage, and a social institution. An appendix works through the exploration/exploitation tradeoff and sketches a mechanism for anxiety as a pathological locking of the system into high-precision exploitation — a regime where the model stops updating because prior confidence is too high to tolerate prediction error.