MemT-1.0: AI Memory INSIDE the Model Is CRAZY (I Tested the Idea)

Author: Prompt Engineer 48 · Watch on YouTube

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: MemT-1.0: AI Memory INSIDE the Model Is CRAZY (I Tested the Idea) · Channel: Prompt Engineer 48 · 11:54 · Watch on YouTube

Topics: AI agents, AI evaluation and benchmarks, Local and open-source models, RAG, search and information retrieval

Summary

The video explains MemT-1.0’s approach to keeping a fixed-size memory inside a model, contrasting it with full conversation history and searchable external notes. It reviews the project’s claims and caveats, including lower storage use, benchmark positioning, hardware requirements, and licensing uncertainty. Since the presenter could not run the real model, they built a toy experiment to compare the three memory strategies, finding that fixed-size memory handled corrections well until its capacity was exceeded. The conclusion is that native memory is promising but not ready to deploy based on the information available, while searchable stores remain useful for retrieving exact details.

Key points

MemT aims to store conversational memory in a fixed-size state inside the model, rather than keeping the transcript or using an external database.
The repository claims roughly 66 times less persistent storage than a full context cache at 128,000 tokens; the presenter emphasizes that this comparison is not against text retrieval.
The video describes MemT-1.0 Nano and Flash as mixture-of-experts models and notes that commercial-use terms for the weights were not yet published.
The repository says textual memory systems still lead overall, especially for long histories requiring detailed evidence retrieval.
The presenter could not run MemT on an RTX 4060 laptop and created a toy experiment instead; it does not reproduce the real architecture.
In that experiment, full history preserved facts when its context window was large enough, while a smaller window eventually lost them.
TF-IDF retrieval used little storage but sometimes returned outdated facts; retrieving more notes improved its results.
The toy fixed-size memory handled corrections well at larger sizes, but performance declined as more facts competed for limited capacity.
The presenter recommends keeping recent chat in a short context window and storing durable facts externally with deliberate handling of corrections.
Open questions include editing or deleting individual memories, model-weight licensing, detailed benchmark results, and practical hardware requirements.

Chapters

Start	Chapter	Summary
00:00	What MemT proposes and what the project includes	The presenter introduces MemT’s model-native memory idea, checks the project’s status and models, and discusses licensing and the three memory approaches.
02:06	How the memory approaches work	The video explains full-history context, searchable external notes, and MemT’s fixed-size latent state, along with the project’s claims and limitations.
05:09	The presenter’s toy experiment	Unable to run MemT, the presenter sets up a comparison using 3,000 messages, changing facts, retrieval, and a simplified fixed-size memory.
07:13	Results and capacity limits	The experiment shows how context windows and retrieval can fail, while fixed-size memory handles corrections at larger sizes but blurs when overloaded.
09:18	Takeaways and practical advice	The presenter estimates the storage scale using a stand-in configuration, outlines unresolved questions, and recommends cautious use of external memory today.

Quotes

Mem T tries a third thing, the memory lives inside the model’s own computation. — 00:00

The repo states that textual memory systems still lead overall — 04:07

A fixed-size memory has a capacity and when you overload it, old things don’t get deleted, they get mixed together. — 08:46

Retrieval is cheap and easy to inspect, but it fails by returning old or wrong notes with a straight face. — 10:21

Model native memory is something to watch, not something to deploy this week — 11:23

Concepts and entities

MemT-1.0, MemT, MemTensor, Qwen, DeepSeek, KV cache, TF-IDF, Gated DeltaNet, RunPod, Prompt Engineer

Full transcript (original language)

00:00 66 times less memory storage. That sentence is on the Mem T repo page and it says that at 128,000 tokens of history, this model stores about 66 times less than a normal full context cache. Now, think about how your AI assistant remembers you today. Either it rereads your entire chat every single time, which gets slower and costs more with every message, or it dumps notes into files and searches them and then answers from whatever the search happened to find. Mem T tries a third thing, the memory lives inside the model’s own computation. No transcript,

00:31 no database. I can’t run the real model on my laptop and I’ll show you why. So, I built the simplest version of all three ideas, gave each one 100 facts, changed some of those facts later, and asked them again after 3,000 messages. One of them forgot almost everything. One gave me old answers with total confidence and one did something I did not expect. >> [music] >> Quick reality check. First, the paper is called Mem T1 towards memory native evolution, submitted October 19th, 2026 by a big team, 74 authors on the archive listing from Mem Tensor. The code went

01:03 public the next day. When I looked, the repo had about 370 stars, 17 forks, and no releases yet, early, very early. Two models, Mem T1.0 Nano, built on a Quen backbone. The repo calls it 36 billion parameters, but only about 3 billion are active for each token, and Mem T1.0 Flash, built on a DeepSeek backbone, 288 billion total, 17 billion active. So, when you hear a 36 billion model, remember it’s a mixture of experts. The memory part on parameters for

01:34 Nano. One more thing, and I think it matters, you’ll see open source in the headlines, read the license. The repository software is Polyform non-commercial, commercial use is not permitted, and the model weights are not covered by that license at all. The page says their terms will come with the releases. So, published and readable, yes. Free for any use, not yet. I’m saying that now, so you’re not surprised later. It doesn’t make the idea less interesting. Let’s look at the idea. Here are the three styles. In plain actions first names second. Style one, keep everything every message stays and the model rereads the lot. That is the full conversation history. In a transformer the model keeps a key and a

02:06 value for every token it has seen. A key value cache so the memory grows with every word. Perfect recall inside the window, but the cost grows with it and when the window is full the oldest things fall off the edge. Style two, write it down and look it up. Every message goes into a file or a database. When you ask something a search finds a few relevant pieces and the model reads those. This is how most memory products work today. Cheap to store, easy to inspect, but it can only answer from what the search finds. Style three is the odd one. One block of numbers fixed size that gets updated after every message. Not a transcript, a compressed

02:36 state. Think of one sheet of paper you keep rewriting. It never gets longer, but writing something new has to squeeze out or blur something old. Mem-T is style three built into the model. The repo describes a native memory block that runs next to the normal attention. A local memory, which is a matrix that changes as the conversation goes and a block with trained projections and gates. An erase gate, a write gate, a forget gate. During a reply the model reads from that memory and adds it into its normal attention. At the end of each turn a gated delta net update writes the new turn into the state. I’m using the repo’s own wording here. The paper has

03:07 the details and I haven’t verified those beyond the page. What I do know from the read me is the headline property. The state stays constant in size and it keeps working without the original text. Constant size, hold on to that word because it is both the superpower and the catch. Here’s what using it looks like straight from the read me. You start a server, open a session and turn on one switch enable memory. After each reply the model writes that turn into its latent memory and then clears the plain text conversation out of context. When you’re done you call save state and get a snapshot. Next week you pass that snapshot back in as the state and the

03:37 model carries on without the transcript. That’s the whole promise in two function calls. The catch, per the read me it needs a Linux server with CUDA and the nano example uses 2GB US. So for now this is lab and server territory. What do the authors actually claim? Three things I could find. One, about 66 times less persistent storage than a full context cache at 128,000 tokens of history. Two for nano, around 1.86 times faster end-to-end at the same length. That’s from the archive abstract page. Three across 11 memory benchmarks, Flash ranks first and nano second among the parametric memory methods they

04:07 evaluated. Read that last sentence slowly. Among the parametric methods, not against everything. And they say so themselves. The repo states that textual memory systems still lead overall, especially on long histories that need detailed evidence retrieval. I like that they wrote it down. It’s the honest version of the story. The model native approach is much cheaper to carry around and it’s competitive, but the old school search the notes approach still wins when you need the exact detail. Also notice what the 66 times is compared against a full KV cache, not against a text file, which is tiny. So, this 66 number doesn’t say better than

04:38 retrieval, it says much cheaper than rereading everything. Those are different claims. Two more things from the abstract. The authors say the models largely preserve general capabilities. So, adding memory didn’t wreck everything else, though largely is doing some work in that sentence. And the repo describes an agentic coding setup where retrieved experience is stored in the native memory and read through memory attention instead of being pasted into the prompt. seen those results, so I’m only telling you what’s claimed. Can I run it? No. And here’s the proof. This is my machine, an RTX 4060 laptop GPU, 8 GB. The readme asks for a Linux server

05:09 with Nvidia CUDA, and the nano example uses two GPUs. The page doesn’t give a VRAM number, so here’s my own arithmetic. 36 billion parameters in BF16 is two bytes each, about 72 GB for weights alone. Even with only 3 billion active, the whole export set still has to live somewhere. I could rent a cloud GPU, but the weights terms aren’t published yet and I’d be paying to find out something the authors already reported in their paper, so I didn’t. This doesn’t contain a single Mem T output. Everything after this point is my own small experiment built to show the tradeoffs, and it does not reproduce Mem T’s architecture.

05:40 Here’s the test, a stream of 3,000 chat messages. Inside it, 100 facts like the access code for vault 25 is 8013. Later, 25 of those get corrected. Half of the corrections use the same words. Correction, the access code for vault whatever is now. The other half are reworded. Update on vault whatever new passcode is, because real people don’t repeat themselves word for word. I also threw in 300 like vault 17 was repainted blue, which shares words with the real facts, but isn’t one. The rest is chitchat. Then I ask for every Vault’s current code at message 500

06:10 1,000, 2,000, and 3,000 10 random streams. Average three contestants’ full history with a context window I can shrink. File retrieval, a real TF-IDF search that pulls the top few notes, and the reader takes the newest hit and a fixed-size memory metrics written with a gated delta rule. That’s the update idea behind gated DeltaNet. Before writing a new value to a key, subtract what’s already stored under it. So, a correction overwrites instead of piling up. Now, honestly, the readers are simple rules, not a language model, so this measures what each memory keeps, not how well a model reads. The

06:41 compressed memory gets a perfect encoder that already knows passcode and code are the same thing. And the cash sizes use a stand-in model config. 48 layers, 8 KB heads, FP16, not memtensors. This is a toy that shows tradeoffs. It is not a replica. Let me show the override because it’s the clever bit. The memory is just a grid of numbers. A key, say Vault 25 code, picks a direction in that grid. To read, I multiply the grid by the key and decode what comes out. To write a correction, I first read what’s currently stored under that key, subtract it, and add the new value instead. One line of code, the old code

07:13 is canceled, not stacked on top of the new one. That’s why corrections didn’t produce stale answers in this memory. But two different keys are never perfectly separate. In a small grid, they overlap a little, and every write nudges its neighbors with 100 facts plus 300 near miss writes. That nudging adds up. Hold that thought for the results. Full history first, give it an 8,000 token window, a normal local setup, and watch at message 500 100% at 1,000 76, at 2,000 18, at 3,000 0. Every fact had scrolled off the edge. Give it 32,000 tokens, and it never forgets. 100% every

07:45 checkpoint, corrections included. Of course, it does. It can see everything. The bill is the point. At 3,000 messages with my stand-in config, that cash is about 6 GB, and it only goes up. Retrieval, tiny storage about a tenth of a megabyte, perfect until the corrections arrive. Then with the top three results, it falls to 92.7% at message 2,000 and 90.7 at 3,000. And look at the kind of mistake. It’s not I don’t know, it’s the old answer. 9.3 stale answers per run. Here’s one, vault 25 original code 8013. Later update on

08:16 vault 25, new passcode is 4855. The question says access code, the update says passcode. The search favors the original wording, finds the old line and answers with it confidently. Let it pull eight notes instead of three and it recovers to 99.9. So, this is fixable with a bigger search and more tokens to read. But, the shape of the failure is real. Retrieval only knows what the search finds. Now, the fixed-size memory, the size of the state is the only dialed. At 256 by 256 numbers, half a megabyte, it holds

08:46 100% at every checkpoint and 99.9 at 3,000. Corrections included because the delta rule overrides half a megabyte against 6 gigabytes. That’s the part I did not expect. Now, the catch, shrink it 228 by 128 and it starts fading as writes pile up. 99.6 88 69 55% at 64 3% by the end. At 32, nothing. There’s no error message, it just gets blurry. A fixed-size memory has a capacity and when you overload it, old things don’t get deleted, they get mixed together. That’s what constant size costs you.

09:18 One more test because a blurry memory raises an obvious question. How many facts can a given size hold? I wrote facts straight into the state, no chatter, and counted how many it could read back. At 256 by 256, it holds everything up to about 200 facts. 99% at 300, then drops to 75% at 500 and 36 at 1,000. Double the width to 512 and it holds 500 perfectly and 89% of 1,000. The rule of thumb in my toy capacity is roughly the width of the state, not its area. Double the width, double the facts. A real trained model may handle this differently and I can’t tell from

09:49 here. But, the lesson stands, constant size means a ceiling and nothing warns you when you hit it. Quick sanity check on that 66, using my stand-in config, 128,000 tokens of cache is about 25 gigabytes. Divide by 66 and you get roughly 380 megabytes. That’s my arithmetic with my config, not MNT’s measured number, so don’t quote it. But, it shows the scale. One history becomes something you could save per user, per session, per project, the way you’d save a document. The repo literally has a safe state call that writes the memory to disk and restores it later without the transcript. That’s the real price here.

10:21 So, what do we take from this? Full history is the most accurate, and it’s the one that costs more with every message. Retrieval is cheap and easy to inspect, but it fails by returning old or wrong notes with a straight face. A fixed-size state is the cheapest to carry, and it holds up well until you overload it, and then it blurs. Real Mem T is not my toy. It learns what to write and what to forget inside a 36 or 288 billion model, and it’s trained on a lot of data. What my toy can tell you is why the authors’ own caveat makes sense when you need an exact detail from a long history. A store you can search

10:52 is hard to beat. My own read, and this is my interpretation, not the paper’s, there’s a practical difference I can’t test here. If memory sits inside a state, you can’t open a file and edit one memory. How Mem T handles forgetting or removing a single fact, I haven’t seen answered on the page, worth checking in the paper. And the open questions are the boring, important ones, the weights license, the per benchmark numbers, which the repo only shows in a figure, whether it runs on hardware that’s normal for anyone but a lab. If you try it on a real server, tell me what you find. If you’re building something today, here’s what this toy suggests. Keep a short window of recent chat because it’s perfect and

11:23 cheap at small sizes. Put durable facts in a store, but handle corrections on purpose. When a fact changes, write it so the new version wins, and retrieve more than three notes, and watch for the mix-up failure because it looks like a confident answer. Model native memory is something to watch, not something to deploy this week cuz the weights terms aren’t out, and nobody has shown it on a laptop. If you’re looking for cheap and reliable GPUs, I use RunPod, and the link is in the description. All the links are in the description, too, the repo, the paper, and my code for this experiment. Question for the comments, would you trust an AI that remembers you, but

11:54 can’t show you the transcript? I’m Prompt Engineer. See you in the next one.