New DeepSeek V4.1 Flash vs Claude Opus 5.5 on a 20,000-Line Project

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: New DeepSeek V4.1 Flash vs Claude Opus 5.5 on a 20,000-Line Project · Channel: Zero to MVP · 05:46 · Watch on YouTube

Topics: AI agents, AI evaluation and benchmarks, AI models, AI safety

Summary

The video compares DeepSeek’s new Flash model with Claude Opus 5.5 on a difficult task in a 20,000-plus-line markdown editor project: keeping links intact when files are renamed or moved. Both models passed all 13 hidden tests, but DeepSeek was much faster and dramatically cheaper. The comparison also uncovered differences in behavior: DeepSeek searched online and added unused alias files, while Opus used the project’s existing parser and made a couple of related fixes. The presenter sees DeepSeek as a strong, inexpensive option for isolated tasks, but remains cautious about sending project code to its servers.

Key points

The task involved automatically updating links when a file is renamed or moved in the Textodus markdown editor.
DeepSeek was tested twice, taking 10 minutes and 7.5 minutes; both runs passed all 13 hidden tests.
DeepSeek cost 8 cents and 6 cents at off-peak rates; the presenter says peak-hour costs would be roughly twice as much.
Claude Opus 5.5 took 21 minutes, passed all 13 hidden tests, and cost $5.45.
DeepSeek searched Git history, used the GitHub API, and attempted code searches for a ready-made solution.
DeepSeek created six unused alias files to cover possible test import names; Opus did not go online.
DeepSeek wrote its own markdown parser, while Opus used the parser already in the project.
In 31 additional edge-case tests, both models passed 30.
Although the weights are open, the model was only hosted by DeepSeek at the time of the video, so code was sent to its servers; local use was described as impractical due to hardware requirements.

Chapters

Start	Chapter	Summary
00:00	The comparison and project task	The presenter introduces the price comparison and describes the link-updating task in a large markdown editor codebase.
01:03	Test results and cost	DeepSeek and Opus both pass all 13 hidden tests, while DeepSeek is substantially faster and cheaper.
02:37	How the models approached the task	DeepSeek searches online and creates unused alias files, whereas Opus checks Git history but stays offline.
04:11	Code quality and conclusions	The models perform equally on additional edge-case tests, but the presenter notes Opus’s extra fixes and raises concerns about DeepSeek’s online access.
05:15	Hosting limitations and final takeaway	The presenter explains that local deployment is not yet realistic and frames DeepSeek as a strong option for isolated everyday tasks.

Quotes

6 cents versus $5.45. — 00:00

Both times it passed 13 out of 13 of my hidden tests. — 02:05

Export the code under several plausible names in case the tests are looking for something else. — 03:08

On my hardest task, the new DeepSeek did as well as Opus 5.5, faster and dozens of times cheaper. — 04:43

Running it locally isn’t very realistic yet because the hardware requirements are too high. — 05:15

Concepts and entities

DeepSeek V4.1 Flash, Claude Opus 5.5, DeepSeek, Textodus, Pi, testes ocultos, parser Markdown, API

Full transcript (original language)

00:00 6 cents versus $5.45. That’s what it cost to solve the same task in my real project. On the left is the new DeepSeek. On the right is Claude Opus 5.5. The test results are the same, but the most interesting part isn’t even the price. It’s what DeepSeek was doing while it worked on the task. More on that in a moment. So, what was released? On October 17th, DeepSeek released a new version of the flash model. Officially, it’s a minor update, but according to

00:32 DeepSeek’s own tests, the model got noticeably better at tasks for coding agents. And on some benchmarks, it beats Claude Opus. But if you look closely, the table doesn’t have the current Opus 5.5. It has the previous generation Opus 5. I’m more interested in comparing it with the current generation Opus 5.5, because that’s the model I use most. Okay, now about the task. I took the hardest task from my previous video where I compared Claude Haiku 5.5 with a

01:03 local quant 3.8 27B. It works with a large code base of my markdown editor, Textodus. That’s more than 20,000 lines of code. Right now, when you rename or move a file, links to it from other notes stop working. They need to update automatically. The model only gets the text of the task. It can’t see the 13 tests I used to check the result. The agent is the same, Pi, with the same settings. The new DeepSeek model has 763 billion parameters, so you can’t run it

01:35 at home. That’s why I’m testing it through the API. Last time, Haiku 5.5 solved this task in 22 minutes for $1.44. The local quant worked for 2 hours and passed nine tests out of 13. Okay, let’s move to the results. To rule out a fluke, I ran DeepSeek twice on the same task. The first time, the model took 10 minutes. The second time, seven and a half. Both times it passed 13 out of 13 of my hidden tests. All the project’s existing

02:05 tests, the linter, and the type check passed, too. And it cost 8 cents and 6 cents. That’s at the off-peak rate. At peak hours, it would be roughly twice as much. But that’s still 10 times cheaper than Haiku 5.5. Now, let’s see how Opus 5.5 handled the same task. It spent longer on it, 21 minutes. Its solution also passed 13 out of 13 tests. No problems there. But it cost $5.45. That’s 70 to 90 times more than

02:37 DeepSeek. It’s good that both models solved the task. But I wanted to know how exactly they worked. And that’s where I got a surprise. First, how DeepSeek worked. The task didn’t say a word about tests. But from the very first steps, DeepSeek decided this was a benchmark with hidden tests. It reasoned about which names the tests would use to import its code. It assumed the task was taken from a real pull request. And in the first run, it went to check It looked through the Git history. Then,

03:08 through the GitHub API, it pulled Texodus’s latest commits, all the pull requests, and searched all of GitHub for an issue with the same title. It also tried a code search. It found nothing. I wrote the task myself, and the tests are in a private repository. So, it did something else. It left six alias files in the project. Each one just re-exports its code under a different name. Its reasoning says it directly, “Export the code under several plausible names in case the tests are looking for something

03:39 else.” Nothing uses these files. My tests check how the editor behaves, not file names. DeepSeek solved the task honestly. But it behaves like a student in an exam, looking for the answers and hedging its bets. And one more thing, the agent went online without asking and sent the task text there. In a sandbox, that’s not a big deal, but on a work project, I’d want to know about it. By the way, Opus also checked the Git history for a ready-made solution, but it didn’t go online and then just did

04:11 the task. Now, about code quality. Here’s another difference. Opus uses the ready-made markdown parser that’s already in the project. DeepSeek wrote its own both times, more than 500 lines. I expected the handwritten parser to make mistakes. So, after the runs, I wrote another 31 tests for edge cases, tables, footnotes, nested lists, code in quotes, and I was wrong. DeepSeek passed 30 out of 31, the same as Opus. On the

04:43 other hand, Opus did more than it was asked. It fixed a couple of related problems in the editor along the way and was honest about what it didn’t check. So, the conclusions. The new DeepSeek model turned out to be really interesting, especially given its price. On my hardest task, the new DeepSeek did as well as Opus 5.5, faster and dozens of times cheaper. I’m not going to replace Claude completely. Opus works more carefully and DeepSeek searching the internet for answers made me a bit

05:15 wary. There’s one more thing. The weights are open, but for now only DeepSeek itself hosts the model. That means your code goes to their servers. Running it locally isn’t very realistic yet because the hardware requirements are too high. But for everyday tasks in an isolated project, DeepSeek now looks like a very strong option. More and more, I’m thinking not about which model is the best, but about which model is good enough for a particular task. This was a first look, one task, and a few

05:46 runs. If you’d like a full comparison, let me know in the comments and subscribe to the channel and to my weekly newsletter about IT and AI. The link is in the description. See you soon. Take care.