Tencent’s Memory Model “Beats Claude Opus 5”. I Rented a GPU to Check

Author: Prompt Engineer 48 · Watch on YouTube

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: Tencent’s Memory Model “Beats Claude Opus 5”. I Rented a GPU to Check · Channel: Prompt Engineer 48 · 11:44 · Watch on YouTube

Topics: AI agents, AI evaluation and benchmarks, Prompt and context engineering, AI infrastructure and hardware, Local and open-source models

Summary

The video tests Tencent’s 30B-parameter High Memory model as a memory writer, using a rented GPU and a community GGUF quantization. The model handles conflicting updates, ignores irrelevant messages, and recalls the latest values well, but in two of five small tests it silently deletes a true memory that was not contradicted. The benchmark lead Tencent reports is small and based on its own evaluation, so the presenter treats the model as promising rather than proven. The video concludes that builders should review deletions and test the model in their own agent setup before relying on it.

Key points

High Memory is a 30B-parameter mixture-of-experts model with about 3B parameters active per token, trained to write and update memory banks.
Tencent reports an average score of 55.49 across four memory frameworks, compared with 54.62 for Claude Opus 5; these are vendor-reported results on a 20% subsample.
The official BF16 checkpoint is about 60 GB; the presenter uses a community Q5 GGUF conversion of about 21.4 GB.
The model is run with llama.cpp on a rented A40 GPU with 48 GB of VRAM, using a simple custom update prompt rather than Mem0 or Memos.
In the first test, changing the location from Seattle to Austin also caused the unchanged back-end engineer fact to disappear.
In the preference test, the model correctly updates the diet but unexpectedly deletes the memory about cooking Italian food.
The model successfully resolves conflicting deadline statements, ignores irrelevant messages, and answers four questions from the final memory bank correctly.
Each memory update takes about 0.1 to 0.6 seconds on the A40; the presenter suggests logging deletions and not deleting memories unless new information contradicts them.
The experiment costs under 30 cents, and the presenter reminds viewers to terminate rented GPU pods to avoid ongoing charges.

Chapters

Start	Chapter	Summary
00:01	What agent memory needs to get right	The presenter introduces the problem of updating a memory bank without accidentally losing facts that remain true.
02:10	Benchmark claims and model setup	The video reviews Tencent’s reported scores, the community quantizations, and the GPU requirements for running the model.
04:17	Renting a GPU and launching the model	The presenter configures a RunPod A40, downloads the model, builds llama.cpp, and starts the API server.
06:49	Five memory-writing tests	A custom prompt is used to test corrections, preferences, conflicting statements, irrelevant messages, and multi-step recall.
10:04	Results, cautions, and experiment cost	The presenter summarizes the model’s strengths and silent deletions, suggests safeguards, and reports the rental cost.

Quotes

Saving facts is easy. Deciding what to keep, what to rewrite, and what to throw away is the hard part. — 00:01

So it isn’t better at everything, it wins on average by less than one point. — 02:10

Nothing in my message asked for that. — 08:24

In two of five tests, an update silently deleted a true fact that nothing had contradicted. — 10:38

I would add a rule to the update prompt that says never delete a memory unless the new message contradicts it, and I would log every delete. — 11:09

Concepts and entities

High Memory, Claude Opus 5, Tencent, Mem0, Memos, llama.cpp, GGUF, RunPod, A40, Repatski

Full transcript (original language)

00:01 Here’s one chat written into memory by Tencent’s high memory model running on a GPU I rented. In the first message, I say I live in Seattle and work as a back-end engineer. A few messages later, I say I moved to Austin. Now, look at the memory bank at the end. Seattle is gone and Austin is there, which is what we want. But the back-end engineer part, which I never changed, disappeared with it. That tiny slip is the whole story of agent memory. Saving facts is easy. Deciding what to keep, what to rewrite, and what to throw away is the hard part.

00:34 And a small mistake there means your agent quietly forgets something true. Tencent released a 30 billion parameter model, about 3 billion active, that does only that job. They say it matches or beats Claude Opus 5 as as a memory writer. So, I rented a GPU, showed every click, and ran five tests. Some of it was impressive and some of it wasn’t. Here’s the model card, high memory, 30B, A3B, Apache 2.0 license. It’s a mixture

01:04 of experts. So, only about 3 billion parameters work on each token. That mixture of experts detail matters for what comes next. The model has 128 small experts per layer, and each token only uses eight of them plus one shared expert. So, the compute per token is like a 3 billion parameter model. But all 30 billion parameters still have to sit in memory because any token might need any expert. Fast to run, big to store. It isn’t a chatbot. Memory

01:37 frameworks like MemZero or Memos have a write path. After conversation, a language model reads it and decides what goes into the memory bank. Tencent trained this model to be that writer first by distilling from a teacher model, then with reinforcement learning that scores the memory bank itself. Why bother? Because without a memory layer, an agent either rereads the whole conversation every time, which gets expensive, or forgets it. With one, you keep short statements like user lives in Austin, and later you retrieve only the few that are relevant. The quality of those statements is decided at right

02:10 time, and that is the step this model replaces. Now the number across four memory frameworks, Tencent reports an average score of 55.49, Claude Opus 5 in the same writer seat gets 54.62. That’s the vendor’s own measurement on a 20% subsample, so treat it as a claim and not a verdict. Look closer though, on the Mem0 setup, Opus is well ahead on Loco M 69 to 60.

02:41 High memory wins on long Mem eval 73 to 66. So it isn’t better at everything, it wins on average by less than one point. One detail I like, Ever Mem was held out of training and the model still scores 56.89 against 54.04. That’s the best hint. It generalizes again by the author’s own numbers. Can I run it here? My laptop has an RTX 4060 with 8 GB. The official checkpoint

03:11 is 30 billion parameters in BF16, roughly 60 GB, not happening. A community member called Repatski made GGF versions. The Q5 file is 21.4 GB and Q4 is 18.2. Important, this is a community conversion, not an official Tencent release, and its own card says a small English test got 23 of 34 cases fully right. Extra storage and duplicates were the common faults. Keep that in mind when you see my results.

03:45 Quick note on that Q5 label, it means the weights are stored at about 5 and 1/2 bits instead of 16, which is how 60-gigabytes becomes 21. On the converter’s own test, the four quantizations from Q4 to Q8 scored almost the same, so I went with Q5, the default that card recommends. With the model file plus working memory, I want a card with more than 24 gigabytes, so we rent. Open the RunPod console, go to pods, and press deploy. The screen that matters is the GPU list. I sort by RAM because that decides

04:17 whether the model fits at all. The A40 has 48 gigabytes and cost 59 cents an hour on the secure cloud. A 4090 is faster, but it’s 24 gigabytes is tight for this file, and it costs more. 48 gives me room. The default PyTorch template is fine. The one change I make is container disk. 30 gigabytes is too small for a 20-gigabyte model plus build tools, so I

04:48 set 60. The total is about 60 cents an hour. Two details on the screen worth knowing. Billing is per millisecond, so a short test costs very little, and the container disk is erased when a pod is stopped. So, for a throwaway experiment like this, I treat the pod as disposable and download the model fresh each time. Deploy. It’s running in seconds, and SSH works with a key already on my account.

05:29 Now, the model. I download the Q5 file and its chat template with Hugging Face client, 20 gigabytes, and it landed in about half a minute. Then llama.cpp. I build the exact revision the GGF card names because older builds don’t know this architecture and this is where things went wrong. So I’ll show it. The build failed because the CUDA compiler wasn’t on the path. One line fixed it and then it compiled.

06:15 Starting the server port 8081 was already taken by RunPod’s own proxy. I moved it to 8090. Small things but they’re the real setup. I used the model card settings, no think mode, temperature zero, and memory writer should be repeatable, not creative. It loaded using about 24 GB of VRAM at a 32,000 token context. A word on why llama.cpp, the official model card shows VLLM with the full

06:49 weights which needs a much bigger GPU. The GGUF file lets llama.cpp run the same model in about 21 GB and it exposes an OpenAI style API that my test script talks to. One note on method, I’m not using mem zero or memos here. I wrote a simple update prompt. The model sees the current memory bank and a new message and returns add, update, or delete operations. So these are my tests, not 10 cents benchmark. Each test works the same way. You see the message go in,

07:19 then the operations the model chose, then the memory bank after each step. I never edit those outputs. The full results file is in the description. Test one, a correction. I say Seattle and back end engineer. Then a rainy weather comment which it ignores correctly. Then I say I moved to Austin. >> The bank now says I moved to Austin with a new job at a startup. Seattle is gone,

07:51 good, but my job as a back-end engineer is gone, too. I only said I changed cities. That’s a real loss of a true fact. Test two, a changing preference. First vegetarian, then I cook Italian on weekends, then I say I started eating fish again, but no other meat. It updates the diet correctly, vegetarian except fish. But look at the

08:24 bank, the Italian cooking memory has been deleted. Nothing in my message asked for that. Second time it dropped a valid fact during an update. Test three, conflicting statements. The deadline is Friday the 7th. My client says Tuesday the 4th, then I tell it my manager decided to keep Friday.

08:55 This one is clean. After the client message, it updates to Tuesday, and after my correction, it deletes that and writes Friday the 7th, one memory latest truth. Test four, forgetting on purpose. I say Rust and Neovim, then a movie quote, a weather question, and a joke. None of those are facts about me. Then I mention my dog Biscuit.

09:26 Perfect here. Quote, question, and joke produce no operations at all. The bank holds Rust, Neovim, and the dog, nothing else. This is exactly what a good writer should do. Test five, the one that matters. 11 messages with changes along the way. Coffee becomes green tea. The one-on-one moves to Wednesday at 4:00. I’m moving to Bangalore and I sell the MacBook for my Windows laptop. Then I ask four questions from the final bank.

10:04 Green tea, Wednesday at 4:00, Bangalore for a team lead role, and the Windows laptop. All four correct and no duplicates. The laptop memory still carries the old MacBook history, which is clutter, but the answer is right. So, what held up? Updating a conflict, discarding noise, and recalling the latest value all worked. Speed was easy, too. Each memory update took between a tenth and six tenths of a second on the A40.

10:38 What didn’t? In two of five tests, an update silently deleted a true fact that nothing had contradicted. For an agent that remembers you for months, that’s the failure that hurts because you won’t notice it. Remember what this is. Five small conversations, a community quantization, and my own simple prompt. It says nothing about the benchmark scores, but it does match what the GG of card warns about. So, I treat this model as promising and check its updates before trusting it with anything important. If

11:09 I were using this in a real agent, I would add a rule to the update prompt that says never delete a memory unless the new message contradicts it, and I would log every delete. I haven’t tested that fix, so take it as an idea to try, not a result. Who should care? Anyone building an agent on Mem Zero or Memos who pays a big model for every memory write. A small open model that does this job at all is the news here. And the part people forget when I finish, I terminated the pod and checked that nothing was left running. A forgotten pod is the real cost of renting a GPU.

11:44 The pod ran for under half an hour or so at about 60 cents an hour. The whole experiment cost under 30 cents. The billing page shows the exact figure once it posts. If you’re looking for cheap and reliable GPUs, I use RunPod and the link is in the description. If you want me to plug this model into a real agent framework and test it properly, tell me which one in the comments.