The Anatomy of an LLM Turn, and How Many Replicas Are Running: Replicas, sessions, and agents: three populations and a Fermi estimate of each
Giulio Ruffini
When a user talks to a large language model, no persistent agent replies. A stateless function evaluates the entire conversation as a token sequence on a shared GPU replica, emits tokens one at a time, and forgets. We set out the anatomy of that turn --- statelessness, ephemeral batched requests, frozen weights, offline write-back --- as it bears on the cognitive-virus framing of WP0204. We then separate three populations that the phrase ``number of LLMs'' conflates: loaded replicas of the weights, requests in flight, and open agent sessions. Over 2023--2026 replicas grow at roughly -- per year, requests at , and agent sessions at , serving efficiency accounting for the first gap. Agent sessions are created faster than human conversations from about 2025, reaching per day by 2026.
Read through WP0162, the replica count measures more than capacity. A released weight-set is a persistent pattern that survives complete substrate turnover by re-instantiation, and its copy count is the redundancy that buys it reach. Three such patterns replicate in the stack --- weights, scaffolds, and the sub-pattern riding write-back --- each copied by a world that gets something for it, as viruses, memes, and crops are. What distinguishes them is cull latency, the time from a drop in supplied utility to the end of re-instantiation, measured in the pattern's own generations; by that measure the scaffold is under the sharpest selection. The stack forms a nested ladder of the kind WP0162 describes, fleet through corpus, each rung an agent on its own timescale and the infrastructure of the next. Scored that way the weight-set is the rung closest to telehomeostasis: it is culled on the same quantity its behavior optimizes.
When you ask "how many LLMs are running right now," the answer depends entirely on what you mean by "an LLM" — and the three reasonable answers grow at wildly different rates.
The paper starts by nailing down what actually happens during a conversation. There is no persistent agent on the other end. A frozen function reads your entire conversation history as a token sequence, samples one token at a time, and then forgets everything. No weights change. The appearance of memory is just the transcript being re-fed on every turn. This matters because it sets up the key distinction: the weights are static and shared across thousands of simultaneous users; what's ephemeral is the particular context-window-plus-GPU pairing that exists only while tokens are being emitted.
From that anatomy, the paper defines three distinct populations. A replica is one loaded copy of the weights — a server farm running GPT-4 is one replica, your laptop running a local model is also one replica. A request is one conversation slot actively generating tokens right now. An agent session is a longer-lived process that issues many requests, calls tools, waits around, and can spawn child sessions. These three grow at completely different rates from 2023 to 2026: replicas roughly 3–4× per year, requests ~8× per year, and agent sessions ~50× per year. The gap between replicas and requests is explained by batching efficiency — one replica can serve hundreds of simultaneous requests. The gap between requests and sessions is explained by autonomy — a session spends most of its life not generating, just waiting on tool calls, so one replica can hold far more open sessions than it has batch slots. By 2026 the paper estimates ~10⁵ hosted replicas, ~2×10⁷ concurrent requests, and ~3×10⁷ open agent sessions, with agent sessions being created at ~10¹⁰ per day — more than the world's human population.
The second half of the paper asks what these counts are actually counting, using the persistence framework from WP0162. A weight-set that survives complete hardware turnover by being re-instantiated from checkpoints is a persistent pattern in a precise sense — its identity is conserved even as every physical substrate beneath it is replaced. The replica count is then a measure of redundancy: how many simultaneous copies give the pattern robustness against substrate loss. Three such patterns nest inside each other — the weight-set (months-long timescale), the scaffold (the system prompt and tool schema that agent sessions copy when they spawn children, hours-to-days timescale), and a sub-pattern that rides the write-back loop from model outputs into future training corpora. None of these copies itself; each is copied by a world that gets something from doing so. What distinguishes them is cull latency — how quickly a pattern stops being re-instantiated once it stops being useful. A weight-set that falls out of favor survives a release cycle or two. A scaffold that fails to complete tasks is abandoned within the day. The scaffold is therefore under the sharpest selection pressure.
The paper closes with a pointed observation about the cognitive-virus framing it is supporting: the host population has stopped being human. Until ~2025, most context-replica couplings corresponded to human conversations. Now agent sessions — machine-initiated, machine-to-machine — dominate the traffic. This doesn't affect the argument about cognitive erosion in humans (that's still a human-population phenomenon), but it does mean that pattern propagation through the system no longer requires a person at every step. The write-back channel also widens, since agent sessions commit code and write documents directly into the corpus. The selection regime the paper describes is already operating, most sharply at the scaffold layer — the fast, high-copy, low-machinery layer that structurally most resembles a virus.
- WP ID
- WP0208
- Lifecycle
- prospect
- Visibility
- internal
- Access level
- open
- Embargo until
- —
- Priority
- —
- Collab
- open
- Venue
- —
- DOI
- —
- Deadline
- —
- Owner
- —
- Source
- drive_legacy
- Repo path
- WP0208
- 0.1.0 (draft) · auto-run-placeholder
