Computing in the Age of Continual Learning
I’ve been making the same point all year: for AI to take the next step, it needs continual learning and world models, not just bigger models. Improvements in reasoning now arrive every quarter. What models still lack is the ability to be better today because of yesterday’s experience. In this piece, I’ll focus on continual learning. Dwarkesh Patel recently captured the problem well in a video reading of his blog post, so that seems like a good place to start.
Patel’s argument comes down to a saxophone analogy. A student who has never played the saxophone walks into a music room, tries to play, fails, and leaves notes about what went wrong. The next student reads those notes before going in, but also has no experience, so they fail too and add their own notes. This procession could continue forever, and no collection of notes would enable the next student to play successfully on their first attempt. At some point, experience has to accumulate in an actual brain. This is exactly how today’s agents work when they leave Markdown notes at the end of one session and read them at the start of the next. By continual learning, Patel means moving beyond this limitation to a system in which real experience gathered during deployment continually updates the model’s weights, improving the model itself.
He lays out eight predictions for what would change if this became possible. A few stand out. As the boundary between training and deployment dissolves, the current regulatory framework of conducting safety evaluations at a single point before deployment would lose its rationale. It would need to give way to regular inspections. Alignment research, which has focused on making a fixed set of weights behave well, would be reframed around keeping a model resilient to jailbreaks and malicious data injection even as its weights change every day. Different instances of the same model would accumulate different experiences, increasing AI diversity. Deployment becoming training would compound the advantages of leading labs, while making it untenable to keep their smartest models under wraps for months.
The most immediately practical point concerns switching costs. Abandoning a model that has spent months learning your organization’s context for a competitor’s model would be like firing an experienced employee and training a new intern from scratch. That is where AI labs would acquire a moat they currently lack. Finally, Patel discusses the economics of inference. Serving sparse models efficiently requires batching thousands of sequences together. In a world where each organization has its own weights, large organizations capable of filling those batches would have an overwhelming advantage. This connects directly to the hardware discussion later in this piece.
So where does the research stand? It broadly falls into three strands.
The first is architectures that update weights during inference. In Titans, Google introduced a long-term memory module that updates during inference, prioritizing information the model finds surprising. Its follow-up work, Nested Learning, goes further, recasting the model and the entire learning process as a system of nested optimization problems operating at different update frequencies.
The proof-of-concept model, HOPE, uses a continuum memory system in which rapidly updated modules retain immediate information while more slowly updated modules hold abstract knowledge. It resembles the brain’s hippocampus and cortex, which learn at different rates. The HOPE architecture diagram above illustrates this memory system operating across multiple timescales.
Within the same strand, Test-Time Training methods have explored compressing context into small sets of weights, known as fast weights, instead of a KV cache. TTT-E2E, released late last year, explicitly reframed long-context processing as a continual learning problem. As the model reads a document, it uses next-token prediction to encode the document’s contents into its weights. At two million tokens, it matched the accuracy of full attention while running 35 times faster. Because context is retained in weights rather than a cache, this technology offers one of the most direct answers to Patel’s call for experience to accumulate in the model’s “brain.”
The second strand tackles catastrophic forgetting through sparse updates. Learning something new at the expense of something old has long been the central enemy of continual learning. Both full fine-tuning and LoRA modify dense representations shared by multiple capabilities, making interference difficult to avoid. Meta proposed sparse memory layers with millions of slots in place of dense feed-forward networks. Sparse memory fine-tuning using these layers showed that a model could learn new facts while substantially reducing damage to its existing capabilities, by updating only the small number of slots relevant to the new knowledge. The principle is straightforward: localizing updates reduces interference. As we’ll see, this is also the most manageable approach from a hardware perspective.
The third strand lets the model direct its own learning. When given new information, MIT’s SEAL generates an output called a “self-edit.” This can include both training data that reformulates the information in the model’s own terms and instructions for hyperparameters such as the learning rate. The model uses this output to fine-tune itself, then uses reinforcement learning to favor the self-edits that produced better results. In effect, it learns how to learn. It resembles a student preparing for an exam by writing their own study notes, and results suggest that larger models are better at this. The accompanying SEAL architecture diagram shows this two-loop process.
Now to the hardware.
Today’s infrastructure for serving AI agents rests on four assumptions. First, weights are read-only, so thousands of sequences can be batched together to amortize the cost of loading them. Second, the only state specific to each user is the KV cache. Third, the absence of gradients allows aggressive quantization, such as INT4 or FP8. Fourth, without backpropagation, relatively inexpensive inference-only chips are sufficient. The three strands of continual learning challenge these assumptions in different ways.
In the first strand, where weights update during inference, the nature of per-user state changes. A KV cache is a predominantly read-oriented state that grows with context length. Fast weights are a fixed-size state that must be both read and written. For extremely long contexts, this can actually reduce total memory use. The tradeoff is ongoing write traffic to high-bandwidth memory, or HBM, and less room for quantization because weight updates require higher-precision accumulation.
Batching is an even bigger problem. If every sequence in a batch has different weights, one large matrix multiplication breaks into many smaller ones, reducing arithmetic intensity. This is the hardware reality behind Patel’s argument about the erosion of batching economics. Early TTT implementations offered an extreme example: updating weights every few dozen tokens left GPU utilization below 5% of peak performance. LaCT, or Large Chunk Test-Time Training, shifted to updates over much larger chunks, ranging from thousands to a million tokens. It raised utilization to 70% while expanding the state to as much as 40% of the model’s parameter count. In this strand of research, designing algorithms around the hardware is fundamental to performance. The accompanying utilization graph shows the difference. And if this state is to persist between sessions, a storage layer is needed to save and reload each user’s weights.
The sparse memory approach, somewhat unexpectedly, starts to resemble recommendation-system infrastructure. Since each token accesses only a few slots in a vast table, the bottleneck shifts from bandwidth to capacity. Memory that handles random access well, tiering into DRAM or CXL-attached memory, and separation along the lines of a parameter-server architecture all become natural choices.
Crucially, all users can share the dense backbone, with personalization represented only by differences in sparse memory slots. This preserves the economics of batching. Systems that keep thousands of LoRA adapters in host memory, page the ones currently needed into GPU memory, and serve multiple adapters within a single batch have already demonstrated the feasibility of this pattern. Personalization through sparse memory is a direct extension of it. That is why this strand creates the least friction with the existing serving stack.
SEAL-like approaches change the fleet more than the chip. They involve a pipeline of generation, fine-tuning, and redeployment, rather than weight updates during inference, so training workloads enter the serving fleet. Optimizer states and activation memory return. Scheduling must accommodate both training and serving. Versioned weight snapshots, rollback mechanisms, and frequent weight distribution across the fleet become routine infrastructure. When weights keep changing, evaluation also becomes an ongoing workload, much like regression testing, rather than a one-time gate. That, too, needs a place in the compute budget.
Several implications cut across all three strands.
Low-precision, inference-only ASICs that do not support backpropagation are less prepared for this future. Write bandwidth, memory capacity, and efficient grouped matrix multiplication become more valuable.
State that could once be discarded at the end of a session becomes an asset that must survive across sessions. Storing and moving persistent state becomes part of serving performance.
Mutable weights can also become a channel for contamination between tenants, increasing the need for isolation.
Continual learning turns buying a model into hiring and developing an employee. What that employee has learned exists as a physical state: weights. Creating, storing, moving, and protecting that state becomes hardware’s new job. Today’s inference infrastructure was built as a read-only factory. It will have to be rebuilt as a workshop that both reads and writes. A hybrid architecture, with most weights residing and being used in the cloud and some locally, may also be necessary.