DeepSeek Finally Fixed Its Biggest Problem (V4.1 Flash 1017 - Fully Tested!)

Author: Prompt Engineering · Watch on YouTube

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: DeepSeek Finally Fixed Its Biggest Problem (V4.1 Flash 1017 - Fully Tested!) · Channel: Prompt Engineering · 13:28 · Watch on YouTube

Topics: AI agents, AI evaluation and benchmarks, AI models, Multimodal models

Summary

The video tests DeepSeek V4.1 Flash 1017, an update that reportedly improves benchmark results while using fewer tokens than the September version. Across autonomous engineering tasks, it builds 3D scenes, runs GPU simulations, trains a quadruped to walk, and handles image- and video-based challenges. The presenter finds that the model can be cheaper per task, not just per token, and recommends using high reasoning effort by default, reserving max for long, open-ended projects. It still trails a frontier model on precise website reproduction, but the presenter considers it strong value for long agentic tasks.

Key points

The 1017 release is the same size as the September model, but has undergone further post-training.
DeepSeek reports benchmark improvements, with Terminal-Bench rising from 31% to 42%.
The model used about a third fewer reasoning tokens at high effort in the presenter’s tests; the reduction was larger at max effort.
Token price alone does not determine task cost: a model that generates many more tokens can end up costing more overall.
The model autonomously created a signed-distance-function dragon, interactive 3D worlds, and a GPU simulation of colliding galaxies.
It wrote a PPO reinforcement-learning algorithm from scratch and trained a quadruped to walk; the full task cost about 22 cents.
For a chaotic double-pendulum video, it tracked the arms across 500 frames and fitted a simulation that later drifted from the footage, as expected for a chaotic system.
It performed well at rebuilding some webpages and designing dark mode, but lagged behind Argon on precise screenshot-based page reproduction.
High effort is the recommended default; max is useful for long open-ended builds where additional polishing matters.
The presenter describes the model as a strong value for long agentic tasks, while noting that it is not a frontier model.

Chapters

Start	Chapter	Summary
00:00	What’s new and why token efficiency matters	The presenter introduces the 1017 update, reviews reported benchmark gains, and explains how fewer generated tokens can lower the cost of completing a task.
04:37	Autonomous builds and GPU experiments	The model takes on 3D creation, interactive worlds, a galaxy simulation, and training a quadruped to walk.
07:44	Vision tests and website reproduction	The presenter tests floor-plan and video understanding, then compares the model’s webpage reconstruction and design work with Argon.
11:23	Reasoning settings and final verdict	The presenter compares high and max effort, recommends when to use each, notes a harness update, and summarizes the model’s strengths and limitations.

Quotes

A model that is half the price, but thinks three times as long, ends up costing you a lot more. — 02:35

it was walking forward after uh, just 46 seconds of training — 07:12

I would use high by default and switch to max only for long open-ended builds — 11:55

it’s not a frontier model, and that shows on the precise front-end work. — 12:56

Concepts and entities

DeepSeek, DeepSeek V4.1 Flash 1017, DeepSeek Harness, Gemini 4 Argon, PPO, Arena web dev leaderboard, Terminal-Bench, DGX Spark

Full transcript (original language)

00:00 Deepseek just released a new version of their flash model. Now, they’re calling it a minor update, but the benchmarks tells a very different story. And the thing that caught my attention the most is how many fewer tokens this model now spends to get there. Deepseek models have always been very competitive when it comes to token pricing, but that never really translated into cheaper tasks because they tend to spend a lot of tokens. But, this release is the first time I think that might actually change. I have been

00:31 running it all weekend in Deepseek harness on real engineering tasks, not just some one-shot games. And so far, it’s have been pretty impressive. And one of my favorite test was asking it to teach a four-legged robot to walk completely from scratch using reinforcement learning. So, here’s the untrained robot, and here it is few minutes into training. In this video, I want to talk about all my learnings from testing this model for the last couple of days. We’re going to look at why a cheap model can still be

01:02 expensive to run, what this one can build on its own, and which reasoning setting is worth paying for. So, let’s start with what exactly is new in this model. So, this is the October release of Deepseek V 4.1 flash, and this new version is called 1017 because of its release date. Now, it’s exactly the same size as the previous September release, but seems like they have done further post training on the model. So, the weights that you see here are going to be very different. According to

01:34 Deepseek, this further training has helped it a lot. It went up on every benchmark in their tables, and the biggest jump is on terminal bench, which goes from 31 to 42%, which actually is pretty impressive given that they’re calling it a minor update. Now, on their numbers, it’s now comparable to OpenAI’s GPT 5.6 soul, which is not OpenAI’s latest model, but this is a flash model at a fraction of the price. Now, these are Deep 6’s own numbers, but

02:05 on Arena web dev leaderboard, where real people vote on which result they prefer, it’s now the number one open model. It’s only eight points behind Gemini 4 Argon there. Again, at a fraction of the price. Now, benchmarks are one thing, but their claim actually is that it’s going to finish tasks faster because it’s a lot more token efficient. So, let’s talk about why that matters because this is where most people get the pricing of these models wrong. The

02:35 price you see on the pricing page is the price per token, but what you actually pay for is the price multiplied by how many tokens the model generates to finish your task. So, a model that is half the price, but thinks three times as long, ends up costing you a lot more. Now, this has been problem with Deep 6 flash models. They are usually token hungry. They produce a lot of tokens. For example, when Artificial Analysis tested the V4 flash back in April, they

03:06 found that it was one of the most token hungry models they have ever tested. The September version of V4.1 flash was even more verbose, and it generated almost twice the tokens of a typical open model of its size to run their benchmark. So, the low price per token was quietly eaten up by all the thinking it was doing. Now, with this release, Deep 6’s own chart shows that the cost of coding tasks at max effort dropping from about

03:36 50 cents to about 20 cents at the same price per token, which is pretty significant. And this only happens if the model is generating a lot fewer tokens. Now, to see if these claims actually hold up, I re-ran some of my benchmarks that I used with the September version with the exact same prompts and settings. So, on high effort, it used about a third fewer tokens for reasoning. And on max, the effort drop was even bigger.

04:07 Now, for my test, the accuracy went up at the same time, especially on object detection, because this has vision capabilities. But, I noticed something really interesting because I ran the tests before DeepSeek had officially announced this new model. The model name in the API didn’t change, and the response fingerprint never changed either. But, the model behind it had clearly changed. Okay, so that is enough on the numbers. Now, let me show you what this model can

04:37 actually do. So, I took the exact same long-running task that I gave Gemini 4 Argon a week ago, and ran every single one of them in the DeepSeek harness completely unattended. Each task is a goal, so the agent keeps working on it. Then, it checks its own output with screenshots, and only stops when it thinks it’s done. So, these are actually long-running agentic tasks, which are fully autonomous. Now, the first one is this 3D test that graphic researcher

05:08 Keenan Crane ran on another frontier model. He asked for a 3D dragon built entirely out of the signed distance function, which is basically describing every shape with math instead of polygons. This is without Blender or any 3D software. So, I gave DeepSeek his exact two prompts and his reference image. So, this is the final render. And every scale, horn, and eyes here is a math function written in a single shader file.

05:38 Now, the output actually looks pretty good. Then, I gave it a three open-ended 3D worlds. One is a procedural planet, uh, a portal physics lab, and a mechanical watch with a real gear train. Each of these is a single web page that it wrote. Um, then it tested those in its browser and kept improving on uh, its own for about an hour. Again, I would say the results are uh, pretty nice here given that it’s an open model that you can potentially run your on

06:10 your own hardware. Now, pages and uh, shaders are one thing, but I wanted um, to see if it can run real experiments on real GPU hardware. So, I gave it access to my DGX Spark over SSH with strict rules about which folders and containers it can touch. For the first task, I asked for a GPU simulation of two galaxies colliding with over half a million stars. Now, in this case it wrote its own simulation, uh, checked that the total energy stayed almost constant, and then

06:40 rendered the full merger. Now, uh, the first time I ran this, my Spark froze completely because, um, I had three heavy jobs running on it at the same time. Now, this was my setup fault, not the model’s, but I expected the model to actually point that out, which it did not. Now, I want you to be the judge of the quality, uh, but this brings us back to the robot from the beginning. Uh, so I asked it to write a reinforcement learning algorithm called PPO from scratch with no RL libraries allowed,

07:12 and teach the four-legged robot and to walk. We are also interested in uh, the time it took um, to make the robot walk properly. So, it was walking forward after uh, just 46 seconds of training, and it hit the target score in under 2 minutes. Now, after 26 minutes of training, it was running at almost 6 m/s. So, the entire job, uh, writing the code, training it, rendering the videos, and building the result page cost about

07:44 22 cents for this specific task. Which is actually pretty incredible. Now, I’m putting together a comparison video comparing this model with some of the other open model releases that we had uh recently. So, if you are interested in that, make sure to subscribe to the channel. So, so far, it’s been pretty good, right? And running the code is one thing, but what happens when the problem only exists inside an image or a video? This is a multimodal model, so I wanted to really test its uh vision

08:14 capabilities. DeepSeek Flash can take images natively, but not video. So, the e-agent has to pull frames out of the video itself and look at them. So, for the first test, I give it a floor plan of an apartment where two of the rooms have no measurements on them and asked it for a uh full 3D walk-through. Now, here’s the thing. It actually worked out the missing room sizes from the rest of the plan and built the uh dollhouse view, a top-down plan, and a

08:46 walk mode, and an automatic tour. Which actually is pretty incredible. Then, I give it a 20 seconds of a realistic chaotic uh double pendulum and asked it to build a simulation that matches the video side by side. Now, this is where it did something really interesting compared to uh Gemini 4 Argon, um which I actually tested last week. So, in case of Argon, uh it quietly measured the arm angles from the video and attached a

09:17 hidden spring that pulls the simulation onto the real footage. Now, DeepSeek on the other hand, uh tracked the two lights on the arms across all 500 frames, then fitted the physics to them, and then let the simulation run completely on its own. So, it matches the real pendulum for the first few seconds, and then it drifts away, which is exactly what a real chaotic uh system should do. So, I think this model is pretty smart. Now, so far all of the results that I

09:48 have shown you shows that this model is actually pretty impressive. But, let’s talk about uh the front-end tests because this is where the gap to a frontier model actually uh shows up. So, I gave it a screenshot of a real website and asked it to rebuild the page from scratch with no access to the original code or images. On my own app’s uh landing page, uh the rebuild is almost pixel perfect. This is, by the way, uh a speech-to-text app for transcription that runs entirely

10:20 locally on your Mac. Check it out if you’re interested. Now, this webpage is pretty simple, uh but on busier pages like NASA’s homepage, it was consistently a little behind Argon. So, this is a nice contrast with the Arena results because people prefer its design almost as much as Argon’s, but Argon is still better at copying a page exactly. So, Argon seems to have much better vision capabilities, and then um it’s able to reproduce uh webpages based

10:51 on images better compared to this new DeepSeek-Flash model. Now, another test uh that I ran was that I gave it only the light version of the page and asked it to design uh the dark mode the site’s own team would ship. And it actually did impressive job here. Now, for the photos on the pages, um it hand-drew every image as SVG. So, the parachutes on the uh NASA page are worth a thousand lines of handwritten vector code. And I’m kind of impressed with these new iterations of the models, uh

11:23 even if you don’t give them images, uh they can reproduce the images using code, which is pretty amazing. Now, let’s talk about uh which reasoning uh setting you should actually use because the model ships with three different effort levels. It’s low, high, and max. Now, in order to compare them, I actually re-ran 20 of the tasks on high to compare it against max. So, on max, it generated about 16% more tokens, but the total time was almost

11:55 the same. On the reasoning tasks and the website rebuilds, max made no measurable difference because high was already perfect on the reasoning, and the pages looks almost identical. But, on the long open-ended builds, max finished the floor plan that timed out on high, and the 3D worlds came out noticeably richer. So, my rule is pretty simple. I would use high by default and switch to max only for long open-ended builds

12:26 where you want it to keep polishing. Okay, so a couple of things to keep in mind. DeepSeek says that they have trained the model within the DeepSeek harness, and they recommend to update it to the latest version, which I believe is 0.2.1 as of the recording of this video. Now, I tested the model in a prior version of the DeepSeek harness as well. I didn’t see any noticeable differences, but that’s something which DeepSeek is

12:56 recommending. So, pay close attention to it. So, with this release, DeepSeek has finally shipped a model that is not only cheaper per token, but also cheaper per task. And seems like they are really pushing for the token efficiency. But, keep in mind, it’s not a frontier model, and that shows on the precise front-end work. But, for long agentic tasks at this price, it’s the best value I have tested so far. So, the pricing war has really begun and

13:28 it’s impacting even the uh Chinese companies as well. Now, if you want to see how a frontier model would perform on these tasks that I discuss in this video, watch this video. And if you have tested this on all your own tasks, please share your results in the comments, especially if you are seeing different token numbers than I did. Anyways, I hope you found this video useful. Thanks for watching and as always, see you in the next one.