CALLIOPE · BCOM

When using a large language model, one types a prompt and receives a response. But what happens between input and output determines the quality, consistency, and limitations of every interaction. This working paper provides a concise technical account of the transformer architecture underlying modern LLMs, explains how the context window is assembled from multiple sources (system prompts, retrieved documents, memory files, and conversation history), and describes how Retrieval-Augmented Generation (RAG) extends effective context beyond the fixed token budget. We then compare three Claude environments---Chat Projects, Cowork Projects, and Claude Code---through the lens of this architecture, showing how each allocates context differently and exposing a fundamental gap: no current environment combines automatic corpus retrieval with persistent working memory. We conclude with a concrete workflow that bridges this gap for iterative scientific paper writing, using the KT research program as a case study.

The transformer has no memory — and once you truly internalize that, everything about working with LLMs becomes clearer.

Every time you send a message, the model doesn't "remember" your previous conversations. It reads a single flat sequence of tokens assembled fresh: system instructions, retrieved document snippets, any memory files loaded from disk, and the current conversation history. That's it. The response is a stateless function of whatever lands in that window. What feels like memory is just text injection done well — or poorly.

The paper walks through the transformer mechanics honestly but without drowning you in math. The key operation is self-attention: every token in the context window can directly reference every other token, regardless of position. A definition at token 2,000 is just as accessible to a sentence at token 195,000 as anything nearby. This is what makes a large context window genuinely powerful — but it also means the window is the only thing that matters. When it fills up, something gets dropped or compressed, and the model simply doesn't know what it no longer sees.

Retrieval-Augmented Generation (RAG) is the main workaround for the size limit. Instead of stuffing 30 papers into the window, you index them as vector embeddings and retrieve only the relevant chunks at query time. Claude's Chat Projects do this automatically and invisibly — you ask a question, the system finds the relevant passages, and they appear in the window as ordinary text. The model doesn't "know" the corpus; it just sees the retrieved excerpts. The limitation: RAG works on static uploaded documents, not on the evolving intellectual state of your collaboration. Decisions made in conversation three are gone by conversation four unless you manually preserved them.

This is the gap the paper identifies and names explicitly: no current Claude environment combines automatic corpus retrieval with persistent working memory. Chat Projects give you RAG but ephemeral sessions. Cowork and Claude Code give you persistent filesystem memory (markdown files that survive across sessions) but no automatic retrieval — Claude has to be told to open specific files. The paper proposes a two-phase workflow to bridge this: bootstrap in Chat to extract structured knowledge from the corpus via RAG into two portable files (CLAUDE.md for stable framework definitions, WORKING_MEMORY.md for evolving decisions and reasoning), then move to Cowork where those files are loaded at every session start and updated by Claude as work progresses. The corpus knowledge travels with you; the intellectual state compounds over time.

The paper is candid that this gap is a product design artifact, not a fundamental limitation, and expects future environments to close it. Until then, the two-environment strategy is the practical answer — and understanding why it's necessary requires understanding what the transformer actually does with your prompt.