Flint: vale trocar o Qwen 35B por esse modelo Americano de 40B?

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: Flint: vale trocar o Qwen 35B por esse modelo Americano de 40B? · Channel: AI ProgBr · 18:50 · Watch on YouTube

Topics: AI agents, Software engineering with AI, Model inference and optimization, Local and open-source models

Summary

The video reviews Flint, a 40-billion-parameter American mixture-of-experts model that activates about 2.5 billion parameters and was designed for agent use. The presenter compares its benchmark results with other models, tests local inference on an M3 Ultra, and investigates why Flint’s specialized Mac engine initially rejects that machine. After using Flint with Open Code to fix the installation check, he tests generation and prompt-processing speeds, finding that performance drops as context grows. He sees promise in Flint for agentic tasks but says more testing is needed before judging whether it can replace Qwen.

Key points

Flint has 40 billion total parameters, activates around 2.5 billion, and uses 256 experts, six of which are activated per token.
Its actual context window is 262,000 tokens, not the 513,000 shown in the presentation.
The presenter says Qwen 8 27B beats Flint in most of the benchmarks shown, while Flint does better in a couple of them.
Flint was trained from scratch by Recursive, and the presenter says development took four months; its weights are open under a 2.0 license.
The presenter runs a Q6 version through LM Studio on an M3 Ultra, initially observing around 90 tokens per second.
A separate Mac engine, which the presenter says can be 1.4 to 2.6 times faster than llama.cpp, rejects the M3 Ultra and even an M4 Max because they are not among its listed supported chips.
Flint and Open Code identify and fix the installation restriction, allowing the presenter to run the engine on his M3 Ultra.
Generation and prompt-processing speeds fall as the context grows; a prompt of more than 100,000 tokens takes 135 seconds to process at 749 tokens per second.
The presenter considers Flint promising for agentic tasks but says he needs to test it further and is unsure whether it can replace Qwen.

Chapters

Start	Chapter	Summary
00:00	Introducing Flint	The presenter introduces Flint’s mixture-of-experts design, intended agent use, context window, and reported benchmark scores.
02:37	Benchmarks, training, and open weights	He compares Flint with Qwen models and describes its development, automated training harness, and open license.
04:10	Model versions and initial local test	The presenter reviews Flint’s available formats and runs a Q6 version on an M3 Ultra, discussing speed and a mistaken answer.
09:24	Testing the Mac engine and fixing compatibility	He investigates the engine’s stated chip requirements, finds that they block even some newer Macs, and uses Flint with Open Code to fix the restriction.
13:04	Generation speed and context effects	The presenter tests the working engine and explains how loaded context affects token generation speed.
15:42	Prompt processing and verdict	He measures prompt-processing performance at different context sizes and concludes that Flint may suit agentic tasks, while further testing is needed.
17:16	Potential use cases and closing	The presenter discusses Flint’s potential, hopes for future speed improvements, and invites viewers to share their results.

Quotes

It activates around 2.5 billion parameters — 00:00

it took 4 months to develop and it was trained from scratch — 02:37

I downloaded the project and put Flint to solve it for me. — 12:01

More than 100,000 tokens were processed, it took 135 seconds — 15:42

I want to do more tests with this model — 17:16

Concepts and entities

Flint, Recursive AI, Mixture of Experts, Eureka Machine, LM Studio, llama.cpp, MLX, OpenCode, Qwen 35B, Mac M3 Ultra

Full transcript (original language)

00:00 Today we’re going to take a look at the new American open model called Flint. Let’s go to the screen so I can show you a little bit. Let’s go then. What is Flint? Flint is a new 40 billion parameter model from an American company based in San Francisco. And despite having 40 billion parameters, it’s a MOE, a mixture of experts. This means that it doesn’t activate all parameters at once. It activates around 2.5 billion parameters, which makes it much faster than dense 40 billion parameter models. And I will tell you that I was expecting something of this

00:31 size, because if we were to quantify it , we would have a total of around 20 GB . And 20 GB will be ideal for Macs with a little more memory; 48 GB would already be enough to load this model. And the new RTX Sparks too, which are actually the main targets of Recursive AI or Recursive Sai, I don’t know, I’m changing everything to ESI. Okay, so Recursive is the company behind Flint. And Flint was specifically designed to run within agents. It has 256 experts

01:01 and activates six of them for each token, but sometimes it makes mistakes, okay? Because we can see here that it has a context window of 513,000 tokens, which is wrong. In fact, its context window is 262,000 tokens. By the way, that’s a spoiler, right? This presentation was done with Flint. Here we have some benchmark data. Terminal Bench 2.1, it scored 54.9 points. In SW Bench Pro V2, 89.7, and in SW Bench Verified, 74.2. It’s interesting to

01:35 have this data here in the presentation , but without context, without a comparison parameter, it loses some meaning. So let’s go to our benchmark graph so we can get a better idea. So here we have Quent. With 6.35 million parameters, in that range, it’s still the absolute king. Even though we have other models that were trained on it, in practice I end up preferring it. Let’s see if it will be different with Flint, because in several benchmarks, the jump is considerable. But this

02:05 doesn’t happen in all benchmarks. In some, yes, but here, it’s already a very small difference. And here it even falls behind. Now, another interesting fact is that the NVFP4 version achieved a higher score than the 16-bit version, at least here in Live Code Bandwidth. In the others, it was very, very close, with about one point difference, not much more than that. Obviously, another model that we’re always keeping an eye on, which is the absolute king below 30 billion parameters, is the Quentas 8

02:37 27B, which our Flint can’t beat in practically any benchmark here. Only in these two does it perform a little better, right? These are the two IFFs, but in all the others, 27B wins. An interesting fact about this model is that it took 4 months to develop and it was trained from scratch, going through the entire process: pre-training, training, supervision, reinforcement learning, all done by Recursive. And the secret here was that they created a large harness, the Eureca Machine,

03:07 which basically automated everything, the whole process. Plus, this model has open weights and a 2.0 license. That means you can run it locally, you can even modify it, redistribute it, you can do whatever you want. And he even does some self-promotion, because in the last slide he launched a “want to master AI” course on aaiprobr.com. The link will be in the description. It’s my AI development course. And he did this because he’s using a skill I’m developing. In fact, if you go to aaiprobr.com, you’ll see that the style of our slide is exactly the style of

03:39 our page here. And that’s not by chance , it’s due to skill. And that’s perhaps one of the biggest secrets. Actually, it’s not a secret at all, but many people don’t know it, okay? Those who are students in my course know that if you understand what you’re doing, you understand the harness, you understand how you structure everything around the model, you make the model more deterministic, less loose. In other words, even if you change the model, you can make it behave in a predictable way. And that’s exactly what I did here with this presentation. So, if you’re

04:10 interested in developing with IAR, take a look at my course. Now, let’s take a look at Hugin Face, because they released four versions of Flint. The first is the 16-bit version with 78 GB. So if you want to run this model here, you’ll need more than 80 GB of V or unified memory. But I’ll warn you right now: even though it’s MOI (mixture of experts), when we get to this size level, with the hardware we have today with unified memory—I’m talking about Macs M3, M4, M5, and also DJX Spark, RTX Spark, which are also computers

04:41 with unified memory—I usually mention these two, forgetting about AMD; there’s AMD too. By the way, if anyone from AMD has one and wants to send it to us to test here, we’ll test it. But well, all these Harders will be good with mixture of experts, but normally they are the more quantized mixture experts. And that’s exactly why they released other 4-bit versions here, because the idea is to actually run it locally. So we have an intgfodar here in Llama CPP, which was made to run on Mac, but normally on Mac we’ll run MLEX

05:11 better. We also have this INT 4, which will be an MLEX format to run on Mac. Now, on Nvidia, RTX Spark, or other cards, the NVFP4 will be the best available option. But of course, we also have, if we go here, in Quantities , we already have the community Quantizations, there’s Dancelot here, with all the sizes you want. So, if you have a 16GB card, you can run a Q2 smoothly. Maybe not with all the

05:43 context, but it’s possible to run it. Since it’s a mixture of experts, you can offload it to RAM and still have usable speed. But well, to make my life easier, what was the first thing I did? I came to LM Studio, downloaded their version, a Q6 version of Flint to run here on my Mac M3 Ultra. With it loaded here, I’ll give you a shout so we can get an idea of the speed. And look, 90 tokens per second. Not bad at all. Let me ask for something a little more elaborate so we can see this speed in a longer text. For example, let’s ask him

06:15 if a hard drive that runs LLMs well will necessarily run image models well? The answer is no. Okay? Let’s see what he’ll say here. So he gave an answer that is a bit longer and, in my view, incorrect, because when we are running LLM models, the bandwidth becomes very important, particularly if we happen to be discussing models that are classified as dense. There’s also an important point in processing power, which is for processing the prompt. Once the prompt is processed and fully cached, you can reuse it very quickly,

06:45 and then basically the entire process comes down to the available bandwidth and the speed of the memory. So you may have hardware that does not have such great processing capacity, but with excellent bandwidth, and it will run LLM models quite effectively. This is the case with the Mac. It’s not that good at either prompt processing or image generation, but it’s good at token generation. When we move over to image generation, then the processing power becomes much more important than the bandwidth capacity. So we can utilize low bandwidth while having higher processing power, and then

07:16 generate images much more rapidly. For example, if you take image, audio, etc. generation, RTX Spark will be faster than on the Mac. This will change because Macs will get better, they will have more processing capacity. The M5 Ultra generates images four times faster than the M3 Ultra, but it only has 50%more bandwidth. In other words, one thing has no direct relation to the other. Wrong answer. Faster, right? 91 tokens per second. In other words, you

07:47 make more mistakes, you make faster mistakes, better or not, I don’t know. Well, if we think that the alternative would be to get it right slowly, maybe getting it right slowly is better. But if you must err, then err slowly; that is good. Better to get it wrong quickly . However, even with 91 tokens per second, I don’t like that speed. Let me explain why. Richard, who is the CEO here at Recurse VII, Recur VSI, whatever, he’s saying in Flint’s introductory post that he can run up to 125 tokens per second on the MacBook

08:18 Pro. And further down we can understand that this MacBook Pro is the M4 Pro, not the M4 Max or the M5 Max, it’s the M4 Pro. I don’t accept, I don’t accept that the M3 Ultra is slower than the M4 Pro. And there are some details here, right? As I explained, for token generation, bandwidth is the most important factor. The M4 Pro’s bandwidth is 273 GB/s, while the M3 Ultra’s is 809 GB/s. So it can’t be

08:53 slower, but it is slower. And then you remember here, they released four different models? Lama CPP is not the fastest way to run a model on a Mac. MLX will typically be faster, although historically, the Q4 versions of MLX usually exhibit a certain degree of quality loss when you run them. I didn’t quite understand why they released a version 24 here, but okay. This one here would be the version, made in MLX for us to run on Mac. But

09:24 there’s another catch here. Going from here to there, I arrived at this repository here, the repository of the hack. Compared to Lama CPP, the hack delivers 1.4 to 2.6 times more token speed per second than Lama CPP. So it achieved those values using its own engine, rather than pure Lama CPP, which is what we were using inside of the LM Studio software. And then I said , I’ll install it. But I had a small

09:54 problem. And the problem is in this line here. The requirements for using the hack are that you have a Mac with an M4 Pro, M5 Pro, or M5 Max chip. What does this mean? This sentence means that the M3 Ultra is not compatible with the hack. This makes no sense. You might think, “No, Igor, it’s only compatible with the M4 and above.” But it’s worse than that. What happens? From the M3 to the M4 there wasn’t any major architectural change. From the M4 to the M5 there’s a small architectural

10:26 change. From the M5 to the M6 we have a significant architectural change. This means that yes, it would run on the M6 and not on anything previous. Since the M5 already comes with ANE, it would even run on the M5 and not on the M4, but once it runs on the M4, there’s no reason why it wouldn’t run on the M3. Now you can be ready to type, Igor: “Ah , are you crazy? You don’t know what you’re talking about. So I’ll show you. I’ll show you something that will leave you speechless. Remember this: M4 Pro,

10:58 M5 Pro, M5 Max. These are the ones written here. And as I told you, what’s written is law for these guys, because they’re crazy. If we go up here and click here, see, support for M4 Max, the M4 Max is better than the M4 Pro. If it runs on the M4 Pro, there’s no reason why it shouldn’t run on the M4 Max. But when you try to install this on the M4 Max, look what happens here. Flint requires you to have an Apple

11:29 with M4 Pro, M5 Pro or M5 Max. We detected M4 Max, so it can’t. I’ve never seen this in my life. What happens? This is crazy. Do you realize this? And I’m not complaining, no, Okay ? It’s something I would definitely do. My makeshift solutions there. But, come on, guys, what’s the deal? Well, once we hit that barrier, there’s only one thing to do. If I’m testing Flint, will Flint solve it? And let me give you a spoiler here. I’m going to type here,”

12:01 hack doctor “. And we have here, look, it didn’t stop, right? Just like it stopped on the M4 Max of the person who’s opening it there. Why? Because I downloaded the project and put Flint to solve it for me. The first time I tried to install it, it gave exactly the same message, only changing the end there to M3 Ultra, which is the machine I’m running it on. So, I took Flint, which was running in LM Studio, put it together with Open Code to identify what the problem was and fix it. It

12:32 identified it and it fixed it. And now we can run our hack here. Now I just have to type” launch open code “in the folder I want to open and it will start Flint and…It’s going to open the open window for us. Great, it’s already open here. Before we type anything, I still haven’t gotten over it. Let’s go back here quickly. I just showed this problem here, but the worst part is the response. Look, someone asked here,” Hack Doctor, it’s blocking the installation on my M4 Max. When can I expect support for the Max? “The

13:04 response was:” Thank you for your question. “We haven’t tested it on the M4 Max yet. We’re looking into it and will update it. What? My God, it works on the M4 Pro. Okay, that’s enough. Okay, okay. Let’s go back here. Let me say hi to our model. Hi, how are you? Look, first prompt there, 10 tokens per second. Let me ask him to tell a story now. Like this, faster. 83 tokens per second. But wait, 83 tokens per second

13:35 is less than 90. What happened? Well, what happened was a small problem called a context window. When the Okay, so here in Open Code, look, we already have 11,000 tokens in our context window. This comes from the system prompt, from the skills that are loaded , with the skill headers. Those who are students in my course know exactly what I’m talking about, but basically this is expected behavior, especially for this engine, for this model, at least for now. The community will probably tinker with this engine even further to make it even better than it is. I mean,

14:07 if this model proves to be really good, right? In my tests, it’s already taken a first step, because it solved the M3 Ultra problem for us. So it is indeed capable of acting genetically, and it is intelligent enough to be able to identify a problem and proceed to solve it. But even here in the End documentation, we see this behavior happening, okay? This is the velocity versus context size map. So it actually starts around 125 tokens. This is in the M4 Pro. But with 8,000 tokens, it already drops below…100. See here? At

14:40 11,000 tokens, it would be around 90 tokens. The problem is that 90 tokens was supposed to be on the M4 Pro, I’m on the M3 Ultra, so it should be faster . For some reason, it’s not happening. And so we can see the speed a little better without the context window full, I’m going to use this tool here. Let me say hi here so we can start looking at this. Here we are. We hit 130 tokens per second. Now we’re starting to speak my language. Even so, it’s still not enough, okay? The M4 Pro hit 125. This

15:11 one, with three times the bandwidth, should be much faster. And I think it can be. But we’ll need to modify Zendnis. Someone will do that soon. Maybe it’ll be me, maybe not. Let’s ask someone to write some code here. Create a clone of Flapbird. And then, I want to see the speed it will take here to …He decided to write everything down. He decided to do it in Python. Okay? 117 tokens per second. A total of 2,000 tokens generated. OK. That’s a good speed. But for a mixture of experts with 2 billion active parameters, less than 3 billion active parameters, I

15:42 expected a little more. I know I’ve said this several times, I’ll stop saying it, but I expected it. Now, the next test that we need to see, let me refresh it here just to clear everything out, is the prompt processing. So, I’m going to ask him to summarize this text here in a paragraph , which is quite long, okay? Let’s see how long it takes him. Look, here’s a result. More than 100,000 tokens were processed, it took 135 seconds, also known as 2 minutes and 15 seconds. And the rate was 749 tokens per second. Here, with this level of filled context

16:14 window, the speed dropped considerably. It went from more than 100 tokens/ second to 37. tokens/second to write this summary. This means that if you are using it programmatically within an agent, as your window size grows, the performance speed of the model will decrease quite drastically. And here we’re talking about both tokens per second and prompt processing, because if we take just half of this book here, a little less, just this excerpt here and ask it to summarize, it processed

16:44 in 5 seconds. Let’s see how many tokens , a total of 8,000 tokens in 5 seconds. So, the rate here is already 1500 tokens per second. So, when we increase the context, the prompt processing also drops. And of course, as we were saying , token generation follows suit, right, 99 tokens per second, because the context window is small. You must be thinking, Igor, aren’t you going to make a game? Well, in this video I wanted to talk more about speed, etc. Because, well, I don’t think you will.

17:16 It’s not suitable for making games. I want to do more tests with this model, but I think games aren’t its strong suit. It will probably perform better in tasks like piloting Leon Hermes and agentic tasks in general. This is a model that I really hope will be very good. I still need to do some more tests. Today it managed to do our presentation and it managed to adjust INne, right? But it wasn’t a very complicated adjustment, but it managed. It managed to adjust it so we could install the hack. Now, probably soon we will have MTP for this model, which is

17:48 something we don’t have yet. So, these speeds will increase and it’s always good to have some more alternatives, right? A model with 40 billion parameters is something you don’t see every day. So, if it could take the place of the Quen with 35 billion parameters, that would be really cool, right? Hopefully, the Alibaba team will launch Quen 4 with 35 billion parameters, a mix of experts, or even a little more, a little less, because with I’m sure if you guys release it, it’s going to be amazing and it’s going to be hard for people to get there. Nobody’s reached 3.5 yet, I don’t think

18:18 this one has either, but I want to experiment more. If you’re already experimenting with Flint, leave a comment below about the results you’re getting and what’s working for you. I want to use it more and I think it will handle some of the things I want to do. If you want to see us running some models here on the other setup with two Arc B70s, check out this link here. And also, if you want to see Flint, leave a comment down below for us. If you want to see the newest Deep Seek, we also made a video about it on the channel, just click here. Now, if you want to fully understand how AI development actually works, then make sure to click

18:50 on this link provided right here. That way you’ll learn how to create your own skills, among many other things. But what I want to tell you is that you won’t be dependent on some magic skill or something magical to solve your problems or on a model that could double in price tomorrow. You’ll know if…It can even turn into models like Flint. I’ll leave it here, see you in the next video.