DeepSeek V4.1 Flash is CRAZY

Author: Prompt Engineer 48 · Watch on YouTube

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: DeepSeek V4.1 Flash is CRAZY · Channel: Prompt Engineer 48 · 10:31 · Watch on YouTube

Topics: AI agents, AI evaluation and benchmarks, AI models, Local and open-source models, RAG, search and information retrieval

Summary

The video reviews DeepSeek V4.1 Flash, comparing DeepSeek’s benchmark claims with the previous version and major closed models, while noting that the results have not yet been independently reproduced. The presenter tests the new search and fetch APIs, reasoning-effort settings, image reading, and a simple game-building task, finding the tools fast and inexpensive in these spot checks. The model appears promising for coding agents and automation, but the presenter cautions that the tests are limited and that self-hosting requires substantial storage and hardware. For most users, the API is the practical option.

Key points

DeepSeek reports improvements over its previous version on all listed benchmarks, including Terminal Bench, Program Bench, Automation Bench, and Apex; the presenter says these figures need independent reproduction.
DeepSeek is competitive on coding and agent tasks in the cited comparisons, while Opus 5 leads on Terminal Bench and the Humanities Last Exam.
The new search and fetch endpoints return ranked search results and page content as Markdown; the presenter’s test calls were fast and the API pipeline worked.
Search supports query operators, and the models endpoint lists a 1,048,000-token context, up to 393,000 output tokens, and text and image input for the new Flash version.
In a single Python coding prompt, low, high, and max reasoning effort all produced code that passed the tests; the presenter advises against paying for max by default based on that one example.
A generated single-file racing game ran on the first try, but it was a simple game and only one attempt was tested.
The presenter’s agent test found third-party providers and returned correct slugs, links, and pricing details, which the presenter checked against DeepSeek’s site.
The presenter estimates the tested agent run and racing-game generation each cost roughly one cent under the stated conditions.
The weights total 763 billion parameters, and community quantized files cited in the video are hundreds of gigabytes; the presenter concludes that API access is more practical for most users.
The review did not independently run the benchmarks, test million-token recall, or reproduce the larger demos; the release also lacks a Jinja chat template.

Chapters

Start	Chapter	Summary
00:00	Release claims and benchmark comparisons	The presenter introduces the release, summarizes its reported improvements, compares benchmark results, and cautions that DeepSeek’s figures have not yet been independently reproduced.
02:06	Testing search and fetch	The presenter explains the new endpoints, their pricing and limits, and tests search results, page fetching, and a model-generated summary.
04:16	Reasoning effort and project generation	The video tests search operators, compares reasoning-effort settings on a coding prompt, and asks the model to build a simple racing game.
06:26	Agent, image, cost, and hardware tests	The presenter evaluates a search-and-fetch agent task and chart reading, estimates API costs, and discusses the model’s context design and demanding hardware requirements.
09:29	Who should use it and limitations	The presenter identifies coding and automation users as the main audience, lists tests not performed, notes setup considerations, and gives a qualified overall assessment.

Quotes

Deep Seek calls this a minor update. — 00:00

the pipeline works, and it works fast. — 03:43

you should not pay for max by default. — 05:20

treat these as spot checks, not a leaderboard. — 09:29

The search and fetch API worked first try, fast. — 10:31

Concepts and entities

DeepSeek V4.1 Flash, DeepSeek, Search, Fetch, Hugging Face, Terminal-Bench, ProgramBench, Opus 5, GPT-5.6, Vercel, OpenRouter, RunPod

Full transcript (original language)

00:00 Deep Seek calls this a minor update. Look at this row. Terminal bench goes from 31.2 to 42.3. The previous version came out 5 weeks ago. That is Deep Seek’s own table, so I’m not going to just repeat it. In this video, I will go through what changed, test the brand new search and fetch API myself, and answer the question everybody asks, can you run this at home? Hi, I’m Prompt Engineer. Let’s start with what actually shipped. This is the announcement from October 17th. Deep Seek V4.1 flash 10 17 1 note.

00:32 Because a lot of posts link to the older repo, the 10 17 1 is the new snapshot, and the older one is where all those downloads are. Deep Seek says reinforcement learning with self-feedback improved end-to-end project work, think building a game or a 3D scene, and it says outputs are shorter, so tasks finish faster and cost less. That second claim is the one to watch because a smarter model that talks twice as much is not a win. The weights are on Hugging Face MIT license with a 1 million token context, taking images and

01:04 text in and text out. Now that table against its own previous version, every row goes up. Terminal bench, 31 to 42. Program bench, 20 to 29. Automation bench, 55 to 60.5. Apex, 65 to 72. Against the big closed models, it is more mixed, and I want to be fair on Deep Seek where Deep Seek lists 77, Opus 5 is 74, GPT 5.6 soul is 73. On Terminal

01:35 bench, they beat GPT, but Opus 5 is still ahead, 52 against 42. And on Humanities last exam, Opus 5 clearly leads, 56 against 42.6. So on coding and agent tasks, it is competitive. On hard knowledge reasoning, the top closed model still leads. Two caveats. These are Deep Seek’s numbers with the harnesses they chose, Claude code for terminal bench and ministravi V2 for deep sub UE and I have not seen independent reproductions yet. Treat this as a strong claim, not a

02:06 verdict. Deep seek also shows four demos, a Forbidden City 3D model, a browser racing game, a portal to recreation in Godot, and a 3D valley scene. These are their examples picked by them. They show the model can finish a long project, but one clip cannot tell you how often it works. I would want to see the failure rate. Now the part I can actually test. Deep seek added two plain endpoints. Search takes a query and returns ranked results with a snippet, a longer summary, and optionally the full page text. Fetch

02:38 takes a URL and gives you the page as markdown. Before you needed a separate search vendor. Now it sits next to your model key, same off, same billing. Each one costs 75 cents per thousand successful calls. You are only charged when you get results back. Failed calls, empty results, and timeouts are free. There’s a limit of five requests a second for each and calls made through deep seek harness, Claude code, or codex are free for now. Per request billing starts on the 1st of November, Beijing time. Let me run it. Query deep seek V4.1

03:11 flash 10 17 release five results. It comes back in about half a second. HTTP 200. The top hit is deep seek’s own news page. Relevance score 2.88, authority three, then their change log, a Bloomberg story, a Versel change log, and the hugging face repo. That ordering makes sense to me. By the way, my very first call came back 401, authentication fails. I had sent an empty key. The error message told me exactly that, which I appreciate. Now fetch on the change log page, 200 again, about a third of a second and

03:43 around 18,000 characters of clean markdown. Then I hand the first 9,000 characters to the model. The model ID is deep seek flash and ask for four bullets, 4 seconds, 2,600 prompt tokens in, 741 tokens out, and 552 of those were reasoning. The summary matches the page, and it even notes that API pricing is unchanged. One caveat, that is one run on one page, not a benchmark, but the pipeline works, and it works fast.

04:16 Search also has query operators, and this is where it gets useful for agents. I restrict to Hugging Face English after the 1st of October, and I get the official repo, then community quantized versions, and a discussion thread. Then I flip it, exclude Hugging Face and DeepSeek’s own site, and I get Open Router, Vercel’s AI Gateway, a Korean news site, and a WCCFTech piece. Same half second each. That tells me the model is already served by third parties, so you may not even need DeepSeek’s own API.

04:50 The models endpoint shows what you can call. The ED DeepSeek Flash is this new version, 1,048,000 context, up to 393,000 output tokens, text and image in. There’s also a DeepSeek V4 Pro and a reasoning effort setting with three levels, low, high, and max. So, I tested effort, same prompt, merge overlapping intervals in Python with tests. Low effort, 2.2 seconds, 471 tokens, 203 of them reasoning. High, 3.9 seconds, 765

05:20 tokens. Max, 3.9 seconds, 688 tokens. I ran the code from all three, and no assertion failed. So, on an easy task, max did not think longer than high. That is one prompt, so do not read too much into it, but it does say you should not pay for max by default. Now, DeepSeek’s big claim, building whole projects. They show a racing game, so I ask for one single HTML file, three lanes, arrow keys, traffic that speeds up, a game over screen, restart on space, neon style, effort set to high.

05:52 68 seconds later, 19,000 tokens, 454 lines of code. I opened it and it ran on the first try. Traffic comes down, speed climbs, and my driver, which is just random arrow keys, crashes at 103 m. Game over screen appears, space restarts it. Is it a Portal 2 recreation? No, it is one simple game. One try and I have not tested how often a prompt like this fails, but the basics, collision, speed ramp, and restart all work.

06:26 Next, the thing search and fetch is actually for an agent. I gave the model two tools, search and fetch, and one job, find which third-party platforms serve this model and what it is called on them. Nobody told it which queries to write. It ran five searches and one fetch, four model turns, 16.7 seconds in total, about 39,000 prompt tokens. It found Vercel, AI Gateway, OpenRouter, and inference.net, each with a source link. Then I checked its answer because a

06:57 confident agent answer is not evidence. The slugs and the links are right. The price it quoted, 15 cents in and 60 cents out per million tokens, matches Deep Seek’s own pricing page for off-peak hours. At peak, those numbers double. It also noted that Deep Seek’s own API does not expose the dated ID. He just called Deep Seek Flash and the pricing page confirms that. Peak hours are 1 to 4 and 6 to 10 in the morning UTC on weekdays. One more because the card says it takes

07:27 images. I sent the benchmark chart from the announcement page as a screenshot and asked for two rows, Terminal Bench and Deep Square, every column in order. 4 seconds, about 1,000 tokens for the image. I compared the answer to the table myself. All 12 numbers match, including the ones for Opus 5 and GPT. So, reading a dense chart works, at least for clean screenshots like this one. So, what does a task cost? That agent run, about 39,000 input tokens and 1,500 output, is under 1 cent in tokens

07:58 off-peak. Add six search and fetch calls, which is half a cent once billing starts, and the whole run lands around 1 cent. The racing game, 19,000 output tokens, is roughly 1 cent, too. Cheap enough that you stop thinking about it until you run it a million times. Why can it do a million tokens at all? DeepSeek’s report title is about pushing the limits of KV cache compression, and the config shows the direction. 64 attention heads, but just one key value head, and a sliding window of 128. Fewer key value heads means a much smaller

08:28 cache to hold in memory for each token you keep in context. I have not measured memory myself, so take that as what the config implies, not a benchmark. Now, can you run it? The weights list 763 billion parameters, mostly stored in 8-bit and 4-bit formats. That is hundreds of gigabytes for the weights alone. My laptop GPU has 8 gigabytes, so no, not here, and not on any single consumer card. Only six of 384 experts fire per token, which saves compute, but all the experts still have to sit in

08:59 memory. The model card gives no hardware requirement, so I will not guess a number, but people have already converted it. One community GB of repo has a Q2K file. Set at 264 gigabytes, Q4KM at 445. My disk has 19 gigabytes free, so even the smallest one is about 14 times what fits. For almost everyone, the practical route is the API. So, who is this for? If you build coding agents or automation, the terminal bench

09:29 and automation bench jumps are the reason to try it, and shorter outputs should help your bill. Test it on your own tasks, because your harness is not theirs. If you self-host, the MIT license is great, but the hardware bar is high. If you just chat, nothing here changes your week. Here is what I did not test. I did not run terminal bench or deep sea myself. I did not fill the million token context to see if it recalls things from the middle. I did not rebuild their portal demo, and I tested one game, one coding prompt, and

09:59 one agent task. So, treat these as spot checks, not a leaderboard. One practical detail, this release has no Jinja chat template. You use the encoding folder or their deep seek recipe library. So, if you load it in your usual stack, expect some setup. If you’re looking for cheap and reliable GPUs, I use RunPod, and the link is in the description. So, the answer better than the last version on deep six own table, competitive on coding agents, behind Opus 5 on hard reasoning, and API only for most of us. The search and fetch API worked first

10:31 try, fast. If you want me to run my own head-to-head on real tasks, tell me in the comments which task you want. Thanks for watching.