An LLM With Memory That Never Stops Growing - Percepta Spotlight Explained

Author: Prompt Engineer 48 · Watch on YouTube

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: An LLM With Memory That Never Stops Growing - Percepta Spotlight Explained · Channel: Prompt Engineer 48 · 10:23 · Watch on YouTube

Topics: AI agents, AI models

Summary

The video explains Spotlight, a Perceptor architecture designed to let a model’s addressable memory grow while keeping the work per token constant. It describes Spotlight’s learned, sparse access to memory cells and reviews results on synthetic recall tasks, language modeling, and long-context tests, while noting that the results are early and use small models and the authors’ own benchmarks. The second half focuses on a transformer with a Python interpreter whose memory stores code and runtime state, can selectively load packages, and can be updated without changing the model’s weights. The presenter sees this as a promising proof of concept but emphasizes its limitations.

Key points

Spotlight is presented as a way to grow a model’s memory without increasing the cost of each token.
Unlike full attention, Spotlight accesses a small, fixed neighborhood of memory cells; unlike fixed-state approaches, its addressable memory can grow with the sequence.
Keys and queries are mapped to addresses in a 2D lattice, and learned addressing determines where information is written and read.
On the reported synthetic recall and long-context tests, Spotlight performs strongly, including on sequences much longer than those used in training.
Short-context language-modeling results are comparable to the baselines, rather than consistently better.
The video describes a Python interpreter implemented inside a Spotlight transformer, with no Python runtime behind it.
The interpreter’s memory can selectively load packages and can be written to or updated without changing the model’s weights.
The presenter cautions that the language models are small, the results are early, and the benchmarks are the authors’ own.

Chapters

Start	Chapter	Summary
00:00	The memory problem and Spotlight’s design	The video introduces the limits of rereading history, summarizing it, and using fixed-size memory, then explains Spotlight’s growing state, sparse access, and learned addressing.
04:07	Benchmarks and long-context results	The presenter reviews synthetic task scores, language-modeling comparisons, and tests of recall at context lengths up to 128K, while noting the results’ limitations.
07:15	A Python interpreter inside a transformer	The video describes how the model executes Python using its own weights and memory, and reports throughput and support for programming features.
08:49	Writable memory and the conclusion	The presenter explains how packages and procedures can be updated in memory, gives an MNIST classifier example, and shares a measured assessment of the work.

Quotes

A model whose memory keeps growing while the cost of each token stays exactly the same. — 00:00

Keys and queries get mapped to addresses in a 2D lattice of cells. — 02:35

And mind you, there’s no Python runtime behind it. — 07:15

The weights never changed and it’s not only facts. — 08:49

So this is a good start. — 09:52

Concepts and entities

Spotlight, Perceptor, Spotlight Memory, Gated DeltaNet, MCAR, Project Euler, Hugging Face, Python interpreter inside a transformer

Full transcript (original language)

00:00 All right. In this video, we’re going to look at a new language model architecture from Perceptor called Spotlight. And the claim is pretty big. A model whose memory keeps growing while the cost of each token stays exactly the same. Now, mind you, I’m reading two blog posts here, both from Perceptor Research, both dated October 2nd, 2026. One is called, “Can LMs grow their own capabilities?” And the other one is called Spotlight Memory. Every number in this video comes from those two posts. So the plan is simple. First the

00:30 problem, then how spotlight works, then the tests, and then the part I find the most interesting, which is a Python interpreter running inside the transformer. So today’s models learn a lot during training, and then they ship. After that, the world they describe keeps moving, but the model doesn’t. Percepta works on decision-m. So think of a hospital where thousands or even millions of new data points arrive per patient per day, or supply chains that change with every order and shipment. Now the model has two options. It rereads the entire history for every new

01:01 token which gets prohibitively expensive or it summarizes the history and then it loses details it might need later. Then there’s rag external knowledge basis and skill files. These do help but you can see that they work by moving the memory outside the model. A separate component decides what to fetch and whatever it fetches has to fit in the context window. So perceptor’s idea is to bring a growing memory inside the model itself as part of the design. Now there’s already a long line of work on memory in neural networks and it always hits the same trade-off. Either the memory has a

01:32 fixed size or the cost of reading it grows with the sequence length. Let’s look at full attention first. It keeps every token so nothing is lost but every new token has to visit the whole prefix. So processing t tokens takes order ts comparisons. Each token costs more than the last. Then we have linear attention and gated delta cost is linear. So that’s good. But they compress the pass into one fixed size state like one page that gets rewritten at every step. So the details get forgotten. So you can see the trade-off here. Attention has

02:04 growing memory but quadratic cost. Gated Deltaet has linear cost but limited capacity and hybrids like Quen 3 Next and Kim linear mix the two but still rely on fixed length attention. Perceptor says spotlight escapes this and they list three properties they care about. Growing state, sparse access and learn addressing. Let me go through them one by one. Growing state means the addressable memory increases with sequence length. Sparse access means each token reads and writes a constant number of memory cells at each step. And

02:35 learned addressing means the model learns where to put things end to end. So here’s how it works. Keys and queries get mapped to addresses in a 2D lattice of cells. A key writes to a small neighborhood around its address and a query reads from the neighborhood around its own. Each cell is a small fixed size recurrent state. It gets allocated on the very first right at that address and future keys can update it. So allocating a new cell adds new storage. That’s how the memory grows and all the cells share the same learned projection and the same

03:05 update rule. So the model learns that rule once and it gets reused no matter how big the memory becomes. Now to keep this trainable they use a differentiable kernel. Attention uses exp of qranspose k which compares every query with every previous key. Spotlight uses an approximation of that with a compact coqued bump. That bump goes smoothly to zero at its boundary. So in one dimension a read or write touches the three nearest cells and in two dimensions it’s exactly 3x3. So nine

03:36 cells that fixed region is yaxis stays constant however long the context gets. Why 2D? Because in D dimensions every access touches 3 to the power D cells. So they keep the routing address at 2D and keep the rich content in separate highdimensional features stored in a delta state inside each cell. So the routing decides which cells and the content decides what’s inside them. In perceptus comparison spotlight costs order T over teps. Attention costs T squ and linear attention is order T but limited by state size and precision.

04:07 Okay. Now the tests first the synthetic ones. MCAR multiquery associative recall a sequence shows many key value pairs and later asks for the values back by repeating the keys. Spotlight gets 0.998 when it’s tested at 256 pairs after training on at most 64 and 0.999 at 131,000 pairs which is a 524,000 token context. Gated Deltanet gets 0.142 and then 0.005. Now the forgetting variant is the interesting one. Half the keys get

04:38 overwritten and the model must return the latest value. Spotlight gets 0.998. Attention drops to 0.748 because the old and the new values both stay in its history. Spotlight just updates the stored association. On MAD, a suite of six synthetic tasks. Spotlight has the highest mean 0.787 versus 49 for gated delta 2, 0.698 for gated delta, and 0.6644 for attention. And on Pathfinder where a 32x32 image is

05:08 fed in as a sequence of 124 pixels, spotlight scores 0.769, gated delta net 2 gets 0.705, gated delta net 0.701 and attention gets 504. Now real language modeling they pre-train at 140 million, 280 million and 670 million parameters on fine web au with the baselines matched on parameter count at 8K context. Spotlight has lower loss than attention at all three sizes. At 670 million, it’s 2.154

05:41 versus 2.203 for attention. It matches gated Deltanet 2 at 255 and gated Deltanet keeps a small lead at 2.142 on LML, which is short context. It’s comparable. 33.1 average for spotlight, 33.8 for gated Delta 2, 33.3 for attention, and 34.1 for gated Delta. No win there and perceptor says the same short context is not where this matters. Now the main test needle in a haststack. The ruler single needle task models

06:12 train at only 8k get tested up to 128k. That’s 16 times longer. Spotlight reaches 93 to 100% recall at 128k. The fixed state models reach at most 5.6% and attention scores zero from 16k upward. Then they do a short stage of 128k long context training. spotlight at 140 million goes from 93.4 to 97.8% and the 670 million model stays at 100. The baselines reach at most 15.6%. So for them that extra training is

06:44 required and for spotlight it’s optional. Loss improves as well at 670 million and 128k tokens. Spotlight gets 1.44 dinats per token about 0.1 lower than both fixed state baselines. And on tira classification from helmet accuracy goes from 63 to 85% as the prompts grow from 8k to 128k tokens. Now to be fair these are early results. The models are small 670 million parameters at most and the benchmarks are the author’s own but

07:15 the direction is clear. Now the second blog and honestly this is the part I like the most. Perceptor built a Python interpreter inside a spotlight transformer. The intelligence module has under 100,000 matrix parameters. It only implements execution semantics and memory control. The memory holds the Python interpreter itself, the core packages, the program’s code, and the runtime state. And mind you, there’s no Python runtime behind it. The model takes Python source as tokens and executes the program auto reggressively

07:47 using only its own weights and memory. It builds on their earlier work, a web assembly computer inside a transformer. You can try it yourself. It’s on hugging face as perceptor AAI/Spotlight VM. You load it with transformers. Give it print oneplus 2 and it gives you three. Now project oiler problems 1 to 3. Throughput holds at 134,000 to 144,000 tokens per second from the shortest trace of 6.5 million tokens to the longest at 47.6 million. So constant work per step however long it runs.

08:19 Perceptter says attention’s work grows with the length of the trace and a fixed state can’t hold an unbounded program’s memory. So they call Spotlight the only architecture that combines general computation with constant work per step. It also handles classes, generators, exceptions, and recursion 2,000 calls deep. Memory is loaded selectively. A program that uses sorted containers and math reads only core Python and those two packages and leaves the rest of the library cold. The memory is over 100 million parameters but each program pays

08:49 only for what it imports. Now the part that answers the title memory is writable give the model a packages source and it writes it into its own memory. That’s their version of pip install. Their example is a tax brackets package for 2023. The model installs it then uses it to compute tax owed. Then the brackets are updated for 2025. The model writes over the old cells. You ask again and the answer changes. The weights never changed and it’s not only facts. The update replaces the old text function as well. So procedures get

09:20 revised exactly like fax. That’s a second route to continue learning besides retraining. Then the biggest example, the model runs an MNEST digit classifier by loading its weights from memory in chunks. That classifier has more parameters than the transformer’s own weight matrices and its predictions agree with the reference implementation on every example. So the ability to read digits comes from the stored network, not from the intelligence module. That’s what perceptor means by intelligence growing beyond its weights. So where does this go? Models that never restart,

09:52 never compact away their past and keep learning methods stored once revised, combined, and reused like software packages or like lemas and theorems in mathematics. My take the retrieval numbers are really strong and the Python demo is a real proof of concept, but the interpreter is hand constructed. The language models are small and the benchmarks are the author’s own. So this is a good start. So in this video we have seen how Spotlight keeps cost constant while memory grows and what that makes possible. Both blog posts and

10:23 the model are linked in the description. Let me know what you think. I’ll see you in the next one.