DeepSeek’s Insane New Architecture

Author: Two Minute Papers · Watch on YouTube

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: DeepSeek’s Insane New Architecture · Channel: Two Minute Papers · 05:00 · Watch on YouTube

Topics: AI infrastructure and hardware, AI models, Open source and software ecosystems

Summary

The video introduces DeepSeek 4.1 Flash, describing its speed, benchmark performance, lower estimated running cost than DeepSeek 4.0 Pro, and native visual understanding. Its key architectural change is CSA2, which shares memory across layers using an encoder-decoder structure and greatly reduces the KV cache. The model has more than 500 billion parameters, and the presenter compares its performance on a paper-reproduction task with other models. The video also notes that 4.1 Flash can use many tokens, while emphasizing the benefits of open research and cheaper AI.

Key points

DeepSeek 4.1 Flash is presented as very fast and able to outperform some competing models on some tests, though not all.
The presenter says it reliably outperforms DeepSeek 4.0 Pro and may cost about a quarter as much to run locally.
The model has native visual understanding, including the ability to use an image of a game menu as a reference for writing a game.
Its KV cache is described as 437 times smaller than V1’s and four times smaller than that of the previous 4.0 Flash.
The paper describes a technique called CSA2 that shares memory between layers.
An encoder creates shared global memory, which the decoder reads from.
The model has more than 500 billion parameters, making it impractical for the presenter to run at home.
The presenter says 4.1 Flash tends to think for a long time and consume many tokens, though it is not very expensive.
The video frames the release and its accompanying research paper as a contribution to open science that can help doctors and scientists.

Chapters

Start	Chapter	Summary
00:00	DeepSeek 4.1 Flash and its performance	The presenter introduces the model’s speed, benchmark results, cost claims, and visual understanding.
01:04	A much smaller KV cache	The video explains the KV cache reduction and introduces CSA2 as a technique for sharing memory across layers.
02:50	Scale and paper-reproduction results	The presenter notes the model’s parameter count and compares several models on reproducing a research paper.
03:52	The catch and the case for open research	The video discusses token usage, the benefits of open research, and Lambda’s tools for running AI experiments.

Quotes

It is incredibly fast. — 00:00

They call this technique CSA2 and they are finally onto something here. — 01:39

DeepSeek 4.1 Flash finally gives us shared memory between the layers. — 02:14

4.1 flash likes to think a lot and burns a lot of tokens. — 03:52

Open science for the win. — 04:26

Concepts and entities

DeepSeek V4.1 Flash, DeepSeek V4 Pro, DeepSeek V4 Flash, CSA2, KV cache, Claude Opus 5, Kimi K3, GPT-6 Astra, Opus 5.1, Lambda

Full transcript (original language)

00:00 The new DeepSeek is here. 4.1 flash and it is wow. Look at that. It is incredibly fast. And I was also stunned by this. Look, it can outperform Claude Opus 5 Kimi K3 on some tests. Not everything. We’ll see about that. But there is a catch. I’ll show you at the end. It also reliably outperforms DeepSeek 4.0 Pro, which is a huge surprise. I mean, that

00:31 system costs maybe 300k dollars to run locally and this one can be run for a quarter of that. Well, that’s still a lot of money, but the tendency is undeniable. This might run in our pocket in a year’s time. It also has native visual understanding. So, finally, you can give it images of an iconic game menu and have it write a game that reproduces it. I’ll tell you more about my other experiments at the end. And this is huge and small at the

01:04 same time. How? Well, now hold on to your papers, fellow scholars. It’s small because of the KV cache. This contains the context and is 437 times smaller than V1 was 3 years ago. But also four times smaller than the previous 4.0 flash. But that was just a few months ago. And you compressed it 4x. That is what? How did they do that? Do we know? Yes, we do. Dear fellow scholars, this is Two

01:39 Minute Papers with Dr. Károly Zsolnai-Fehér. Why? Because this is an open model and there is a free research paper explaining [music] it. I am not an expert, just a student, but I’ll try to break it down. They call this technique CSA2 and they are finally onto something here. You see, a neural network is given by a lot of layers of neurons, and as information is propagated through these layers, each of them has its own KV memory. But, not here. DeepSeek 4.1

02:14 Flash finally gives us shared memory between the layers. Not everyone has to remember everything. No. They share the memories. [clears throat] It’s a nightmare to pull off well, because different layers need different views of the same history. And if we pop the hood and look closer, hohoho, look, an encoder-decoder structure. The encoder creates a shared global memory, and the decoder reads from it. And all that is what finally gives us a

02:50 much smaller KV cache, which is the bane of my existence, because it needs too much video RAM. Incredible leap forward. Now, I also said this is huge, more than 500 billion parameters. So, I have no chance to run this at home whatsoever. And when I tried to reproduce this beautiful honeycombing paper, GPT-6 Astra nailed it. Absolutely stunning. Ooh.

03:21 Opus 5.1, not as stunning, but the physics is quite formidable. Rendering a bit less so. Now, this one. Well, we still need to get a couple more papers down the line to match that, but it will happen. Of that, I am certain. And you see, I am here to show you the truth, not just believe the headlines. Now, also, since YouTube stopped recommending my physics simulation papers, which breaks my heart, at least I get to show you this, which fills me

03:52 with joy. Subscribe and hit the bell to see more. Also, the catch, 4.1 flash likes to think a lot and burns a lot of tokens. Fortunately, it is not that expensive, but man, that’s a lot of tokens. But, if you have the hardware, you get to run all this for free. I don’t, so I run it through the API or lambda, but given that every time Deep Seek publishes a new paper, AI gets cheaper for all of us. So, thank you so much for that. Huge contribution to

04:26 humanity. This is going to help a lot of doctors and scientists do their work. Open science for the win. What a time to be alive. I use lambda to reproduce AI research papers often in minutes. It’s also great to train your own models or fine-tune an existing one. Run inference or text to image or video, easy peasy. Running a Deep Seek chatbot or agent, super fast, super reliable. Lambda gives you powerful Nvidia GPUs to run your own experiments. I test ideas from the

05:00 papers I cover, and moments later, results. Love it. Seriously, try it out now at lambda.ai/papers.