Onde o Haiku supera GPT Luna e DeepSeek

Author: Cortes do Rafael Quintanilha · Watch on YouTube

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: Onde o Haiku supera GPT Luna e DeepSeek · Channel: Cortes do Rafael Quintanilha · 06:21 · Watch on YouTube

Topics: AI evaluation and benchmarks, Model inference and optimization, AI models

Summary

The video presents Haiku as a strong, low-cost model that outperforms its previous version, GPT-4o, and Gemini 1.5 Flash in the cited benchmarks, while generally trailing Sonnet. The speaker stresses that Haiku is best suited to high-volume, well-scoped tasks and agents, rather than long or complex coding work, especially because costs rise sharply beyond 100,000 tokens. A comparison with GPT-4o Mini and DeepSeek V3 highlights the role of cached input pricing and provider reliability in determining actual costs. The video also emphasizes Haiku’s speed and low latency, concluding that it belongs among the good, cheap, and fast models alongside DeepSeek.

Key points

The benchmark comparison presented shows Haiku beating its previous version, GPT-4o, and Gemini 1.5 Flash in every listed category, including agentic coding.
Haiku still trails Sonnet on most tasks, but the speaker says it costs 20 times less.
Prices increase fivefold when usage exceeds 100,000 tokens in the window.
Haiku is positioned for high-volume, well-defined tasks and agents, rather than long or highly complex tasks.
The speaker cautions against making Haiku the primary model for coding tasks, even though it can perform well when the scope is defined.
In the OpenRouter comparison, Haiku and GPT-4o Mini have the same price up to 100,000 tokens, while DeepSeek V3 through Fireworks is described as nearly three times more expensive.
Very cheap cached input can make Haiku cheaper in a weighted average, although the speaker notes that usage patterns affect the comparison.
The speaker reports Haiku at 154 tokens per second, compared with 28 for Luna, and highlights its lower latency.
DeepSeek V3 through Fireworks is reported at 50 tokens per second, while another DeepSeek provider reaches 147 tokens per second.
The video concludes that Haiku and DeepSeek offer similar speeds and prices in the comparison, placing Haiku among good, cheap, and fast models.

Chapters

Start	Chapter	Summary
00:00	Haiku’s benchmark performance and price	The speaker highlights Haiku’s benchmark results and low cost compared with Sonnet, then explains the price increase above 100,000 tokens and the model’s intended use.
02:07	Cost, performance, and API credits	The speaker discusses a benchmark chart, says Haiku is not intended for programming, praises Sonnet’s cost-benefit, and mentions API credits.
03:12	Comparing Haiku, GPT-4o Mini, and DeepSeek	The speaker compares model pricing on OpenRouter, explains the choice of a reliable DeepSeek provider, and discusses cached input and weighted-average costs.
04:47	Speed and latency	The speaker compares Haiku’s speed and latency with Luna, GPT, and DeepSeek across providers, concluding that Haiku joins the group of good, cheap, and fast models.

Quotes

it’s 20 times cheaper than Sonnet. — 00:32

Haiku is not a model for you to run very long tasks. — 01:04

Haiku is not a model for you to be using for programming — 02:40

Haiku is five times faster than Luna. — 04:47

Haiku has joined this little group of good, cheap, and fast models. — 06:21

Concepts and entities

Haiku 3.5, Sonnet, GPT-4o, Gemini 1.5 Flash, GPT-4o Mini, DeepSeek V3, Luna, Anthropic, OpenAI, OpenRouter, Fireworks, AWS, cached input, tokens por segundo

Full transcript (original language)

00:00 Let’s start by looking at Haiku’s performance, because this really surprised me. Folks, notice that the comparison they make is against the previous version of Haiku, the 3.5, and also against GPT-4o and Gemini 1.5 Flash. And basically, it beats them in every single category, okay? Including agentic coding. If we use the benchmark comparison, it’s far superior, alright? It’s true that it still lags behind Sonnet in most tasks, but folks, it’s

00:32 20 times cheaper than Sonnet. So, it’s a model that costs 5%of the price of Sonnet. So, man, your plan using Sonnet and Haiku, you’ll probably be able to use it all month long, alright? So, this left me quite satisfied and happy with what Anthropic has done. And looking at the pricing, look, we have cache reads and input at 10 cents and 50 cents, okay? So here, it’s 20 times cheaper in both cases. Now, there is a catch, okay? If you go

01:04 over 100,000 tokens in the window, then the price increases significantly, it quintuples. Got it? So it goes to $ 2.50 for output and 50 cents for input. “Oh, Rafael, an Anthropic trick.”“I knew there was a catch to this.” Guys, Haiku is not a model for you to run very long tasks. It’s not meant for you to take a super complex task and put Haiku to work on it. No, actually, this is a model for you to execute higher-volume tasks, as they said, tasks with a well-defined scope and for

01:35 your agents. So it’s not meant for you to really be doing much more complex tasks, okay? That’s not the point of the model, and its price is telling you that. So, don’t make Haiku 3.5 your main model for coding tasks and think: “Oh, but it’s getting much more expensive.” That is not the goal, although it might possibly perform well , right? Since it also performs well on various tasks, especially when the scope is better defined. Got it? Now let’s take a look at this benchmark chart using Haiku 3.5. And here we have

02:07 Sonnet with the price difference, right ? First, look at how the price difference is interesting, isn’t it? The graph is shifted, right? The graph is pushed to the left, meaning it’s getting cheaper, right? And Haiku is this red one here. You see that Haiku at high X ends up being more expensive than Sonnet, right? But you see it has a performance similar to Sondium, doesn’t it? So the truth here, guys, is that Haiku is not a model for you to be using for programming, okay? It is a

02:40 model for you to use for other things that we will see here. Alright? And Sonnet really has an excellent cost-benefit here as well. That also needs to be said, right? Really, really cool. Well, here they are talking about API credits, right? So, those who are on the $ plan will receive $ in the API . Those who are on the $ 200 plan will receive $ 2 in the API. And if you are on a team, you can receive up to $ 500, depending on how much you pay per person, okay? So this was really,

03:12 really cool. Now I wanted to show a comparison here on Open Router between Claude Haiku 3.5, GPT-4o Mini, and I also added DeepSeek V3. Which is a model typically considered a good, cheap, and fast model, okay? If we compare these three models, what do we have here? Okay? First, I set the provider for Haiku as Anthropic, for GPT-4o Mini as OpenAI, and for DeepSeek V3 I used Fireworks, which is a provider I trust, even if it isn’t necessarily the cheapest, okay? But

03:44 it’s no use just picking by price, because since it’s open source, you will see several providers offering it here. Some will be cheaper indeed, okay ? But they aren’t as reliable. So, to have a fairer comparison, I chose a provider that is very stable, which is Fireworks. And if we look here, guys, look at the difference, okay? Haiku and Mini are now at the same price, up to 100,000 tokens, right? And DeepSeek is already, look, almost three times more expensive, okay? But the difference is that because of the cached input which is absurdly cheap, it ends up, in the

04:16 end, in the weighted average, being even cheaper than the alternatives. And Haiku ends up being a little bit more expensive than Mini. Now I repeat, guys , this is not the model for you to be doing many iterations. So, in fact, it will end up being more expensive in terms of weighted average, but in practice, that might not happen often because you’ll typically use it for tasks where caching matters less, especially since the price increases after a while, okay? Now, folks, one thing I wanted to draw your attention

04:47 to is right here, the TPS, the speed, right? Why? If you use Haiku through the API on the cloud platform, you’ll be able to use it at 154 tokens per second at the moment. By comparison, Luna is currently at 28 tokens per second. We’re talking about five times slower, okay? Haiku is five times faster than Luna. And look at the latency, six times, or five times better latency, right? It’s faster in

05:17 speed and faster in latency too. Oh, Rafael, but it’s not fair to compare OpenAI’s subscription with Haiku on AWS . Yes, it is fair. Why? Because I don’t think you necessarily should use Haiku on your subscription. Maybe if you’re on that dollar plan, fine. But if you’re using it via API, and now those of you on the Max plan will have at least some dollars to spend per month, you’ll be able to have very, very generous usage of Haiku via API, man, with a TPS of 154 tokens per second.

05:50 That is excellent. Over at GPT, sure, you can connect with your OpenAI account, okay? You won’t pay anything extra either, but look at the speed difference, folks, it’s huge, okay? And by comparison, DeepSeek V3 4.1 Flash over at Fireworks, man, it’s at 50 tokens per second, okay? Now, it is true that there are other providers here for DeepSeek that deliver much faster, right? DeepSeek itself, right? It will deliver here, look, 147 tokens per second and very good latency. So, in fact, if we’re going to make this comparison here, we have DeepSeek and Haiku at similar speeds and similar

06:21 prices. So Haiku has joined this little group of good, cheap, and fast models.