CALLIOPE · BCOM

This work introduces the Agentic Compressor, a theoretical framework proposing that model construction itself must be treated as an action-selection problem within intelligent systems. Operating within the Kolmogorov Theory (KT) framework—which decomposes algorithmic agents into a Modeling Engine, Objective Function, and Planning Engine—the paper identifies a structural gap in existing formulations: the process by which a Modeling Engine updates itself has been treated as fixed wiring rather than as a space of plannable operations. The framework addresses this gap by equipping the Modeling Engine with an explicit internal action space, comprising operations such as hypothesis simulation, submodel revision, reparameterization, and theory unification, each evaluated by a compression-gain objective that balances model improvement against computational cost. This yields a dual-loop architecture in which an internal planning engine acts on the model while the external planning engine acts on the world, with the two loops coupled through shared data and shared objectives. The Transformer attention mechanism is identified as a primitive, implicit instantiation of this idea, and large language models are reframed as Turing machines with pre-loaded world models whose full ME/OF/PE triad can be activated via prompting. The analysis concludes that closing the gap between current machine learning and human-level model construction requires elevating internal model-building operations to first-class status under a unified compression-gain objective.

Learning itself needs to become a planned activity, not just a reflex — that's the core bet of the Agentic Compressor.

Current AI systems learn by ingesting data, computing a loss, and updating weights. This is powerful but passive: the system has no say in how it learns. It doesn't decide which hypothesis to probe, which part of its model to challenge, or which internal simulation is worth running. The Agentic Compressor argues this is a structural gap, not just an engineering shortcoming. Within the Kolmogorov Theory (KT) framework — which describes intelligent agents as a Modeling Engine (ME) that builds compressed world models, an Objective Function (OF) that assigns value, and a Planning Engine (PE) that selects actions — the model-update process has always been treated as fixed wiring. This paper proposes making it a first-class action space instead.

The idea is to give the Modeling Engine its own internal repertoire of operations: simulate a hypothesis, compare two candidate model structures, reparameterize the latent space, unify two submodels, allocate more compute to a hard subproblem. Each operation is evaluated by compression gain — how much it reduces the total description length of model plus residuals, minus the computational cost of running it. An internal planner then navigates this space strategically. The result is a dual-loop architecture: one planning loop acts on the world, another acts on the model itself. A physicist deciding "what if the potential is quadratic?" and then checking it against data is the paradigm case.

The paper identifies Transformer attention as an already-existing, if primitive, instance of this idea. During training, attention patterns selectively route gradient signals — some parameters get updated strongly, others barely at all, depending on context. That's selective model revision. But it's implicit, covers only one operation, and has no meta-planner reasoning about which attention pattern would yield the best compression gain. The Agentic Compressor generalizes this to a full repertoire under a unified objective.

On LLMs specifically, the paper sharpens a position: an LLM is not a passive component that becomes an agent only when wrapped in external scaffolding. It is better understood as a Turing machine with a pre-loaded world model — its parameters are a massively compressed model of its training distribution, operationally verifiable via arithmetic-coding-based compression. What it lacks in isolation is an Objective Function and Planning Engine, but both can be supplied via the prompt. Chain-of-thought reasoning is already a primitive form of internal planning over that model; a well-designed harness amplifies it. The missing piece, the paper argues, is a dedicated internal planner with an explicit compression-gain objective rather than one implicit in autoregressive token prediction.

The broader implication is pointed: if the gap between current deep learning and human-level model construction lies in the absence of internal planning for model building, then scaling data and parameters alone won't close it. The architectural innovation needed is to elevate model-building operations to first-class status within the learning loop — making the Modeling Engine an agent in its own right, whose world is the space of possible models.