Flint - The First RSI Coding Model?

Author: Sam Witteveen · Watch on YouTube

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: Flint - The First RSI Coding Model? · Channel: Sam Witteveen · 18:37 · Watch on YouTube

Topics: AI agents, AI evaluation and benchmarks, AI models, Local and open-source models

Summary

The video reviews Flint, a 40-billion-parameter mixture-of-experts coding model with about 2.5 billion active parameters per token, designed to run much faster than dense models such as Qwen 3.8 27B. It compares Flint’s benchmark performance, architecture, hardware requirements, and hands-on behavior, noting that it approaches dense-model results on some tests but trails on others. The creator also examines Recursive’s claim that its Eureka Machine helped develop the model, while emphasizing that there is limited public detail to verify how the system works. Flint appears promising for fast local coding, especially on powerful Macs, but its speed and usefulness depend on the hardware, quantization, and agent harness.

Key points

Flint has 40 billion total parameters, with about 2.5 billion active per token; it selects six experts from 256 and uses one shared expert.
The low active parameter count supports faster inference, while the total parameter count determines whether the model fits in memory.
Its architecture uses sliding-window attention in most layers and full attention in 14 layers; the model supports up to 262K context and 65K output tokens.
Flint is released under the Apache 2.0 license, with bfloat16, INT4, GGUF, and NVFP4 versions.
It is behind Qwen 3.8 27B on some coding benchmarks, but offers a substantially higher speed potential as an MoE model.
Flint is available through tools including Ollama, LM Studio, llama.cpp, vLLM, and SGLang; Recursive’s rec engine provides the headline Mac speed.
Recursive says its Eureka Machine helped explore architecture, quantization, and GPU kernels, but no paper describing the system has been published.
The Eureka Machine is described as a parallel loop in which agents propose ideas, run experiments, and build on promising results; reliable evaluation is a key challenge.
The creator’s tests show fast generation and competent tool use, while Recursive’s guidance suggests breaking larger tasks into smaller steps and choosing an appropriate harness.
The NVFP4 version is easier to run than the full-precision version; the creator says even an RTX Pro with 96 GB could not run the full bfloat16 model.

Chapters

Start	Chapter	Summary
00:00	Why consider Flint?	The video introduces the speed limits of a dense coding model and presents Flint as a faster MoE alternative.
01:35	Flint’s architecture and benchmarks	The creator explains Flint’s experts, attention design, license, model releases, and benchmark trade-offs.
04:13	Running Flint and Recursive’s background	The video covers available inference tools, recommended agent guidance, and the people behind Recursive.
08:27	The Eureka Machine and model development	The creator describes Recursive’s agent-driven research loop, its evaluation challenges, and the reported design choices behind Flint.
13:17	Hands-on tests	The creator demonstrates Flint’s generation speed, reasoning style, essay review, and tool use.
16:33	Hardware requirements and verdict	The video discusses GPU optimization and memory needs before summarizing who may benefit from Flint and what to watch for next.

Quotes

it can hit 130 tokens a second. — 00:32

There are 256 experts. It picks six of them per token, and then there’s one shared expert that’s always on — 01:35

the hard part wasn’t coming up with the ideas, it was coming up with the judge — 09:00

it’s quite verbose in the thinking. — 13:49

you take quite a hit, when you do that as opposed to going for a fully dense model. — 18:37

Concepts and entities

Flint, Recursive, Eureka Machine, Mixture of Experts (MoE), NVFP4, Richard Socher, Qwen3.8 27B, Pi, llama.cpp

Full transcript (original language)

00:00 Okay, so on the channel, I’ve covered the Qwen 3.8 27B model quite a lot. We’ve looked at variations of it, and I gotta say, it’s still one of my favorite models. the problem, though, for a lot of people with that model is just speed. Because it’s a dense model, when you’re decoding, you’ve gotta go through all of the layers now on a MacBook Pro with an M4, it does about 13 tokens a second, and I guess that’s fine for chat, but it’s kind of painful for anything that you want to do with agents where you’ve got dozens of calls, you’ve got a loop going,

00:32 et cetera So in this video, I want to talk about a new model that’s trying to take on these mid-level coding models, but do it from the point of an MoE rather than a dense model. So this is Flint running on that same MacBook that I was talking about before, it can hit 130 tokens a second. It’s roughly 10 times faster than the Qwen dense model And on some of the coding benchmarks it’s getting really close to the same kind of performance. On others though, it’s kind of well behind. So we’ll get into that but I gotta say before we even do that, That’s not

01:04 even the most interesting part here. The company that made this, Recursive, say that they built the whole model from scratch in under four months With basically a very small machine and a set of agents and they did it with their automated research system, which they call the Eureka Machine, which helped design the architecture, the quantization, and even the GPU kernels So let’s dig into whether Flint is actually any good and also what do they mean when they actually say an AI helped build this? Okay, so what is Flint?

01:35 It’s a mixture of experts model with 40 billion parameters in total, but only about two and a half billion active for each token. There are 256 experts. It picks six of them per token, and then there’s one shared expert that’s always on And this is the thing that really matters for running something like this locally on a laptop or on a desktop box. Your speed is mostly limited by your memory bandwidth. So how many weights do you have to read for every token?

02:06 The active count here is really what predicts the speed. the total count or the number of parameters in total, is what decides whether it’s gonna fit at all So I guess the two questions really to think about here is one, is 2.5 billion parameters too small, to basically have for your active parameters? And then also is 40 billion parameters too big for many people to be able to fit this on their GPU? there are some other interesting things about the architecture of this model as well. So most of the layers, 35 of them… are using what’s called a sliding

02:40 window attention of about 500 tokens And then only 14 of the total layers actually use a full attention system. So that keeps long context a lot cheaper. And they’re documenting that out of the box this can do 262K of context with up to 65K output tokens. Another interesting thing here is reasoning is always on now, one of the cool things here is that this is a fully Apache 2 open license. it’s not gated. They’ve released four official repos on Hugging Face.

03:10 You’ve got a bfloat16 version. you’ve got an INT4 version. They’ve got a standard GGUF quantized version, and they’ve got an NVFP4 one which is what I’m gonna mostly show in here Now, if we just go for a quick look at the benchmarks against other mixture of experts models in its class, things like the Qwen 3.6 35B, which is a model that a lot of people wanted to see updated for the 3.8 and it just never came. That’s the mixture of experts model that’s sort of closest to the 27B and

03:41 then other open MoEs like Laguna XS. Uh, Flint’s leading pretty much on everything, and it’s basically tied with Qwen 3.6 on LiveCodeBench On instruction following it seems to be beating out all of its rivals there when we compare it straight up against the Qwen 3.8 27B, it does lose, right? On Terminal-Bench, it’s 55 versus 73, and it’s behind both on SWE-bench numbers and the LiveCodeBench so it’s not that this is the best coding model out there.

04:13 It’s that, this is a model that gets really close to the dense model performance, but is an MoE and can run much, much faster and honestly, this chart makes a better case than any of the benchmarks here. You can see sort of higher scores up here, more speed out here, and looking at this, Flint is sitting out all on its own over there All right, so if you want to run this, you’ve got a whole bunch of different ways you can do it. It’s already out in Ollama. It’s also in LM Studio and llama.cpp. and there’s vLLM and SGLang support if you want to serve it properly

04:46 Recursive themselves have made their own engine called rec, and that’s where the 130 tokens a second is actually coming from But it seems at the moment that will only install on a Mac, right? and a pretty powerful Mac at that. Unfortunately, I don’t have one of those, So the Mac footage that you’ve been seeing is actually from Recursive’s own launch video I’m gonna mostly show the NVFP4 version running on an RTX PRO 6000 Now, if you want to try and run this yourself, Recursive’s own blog says that Flint may actually need more guidance than bigger closed coding models so

05:18 I guess the goal there is when you’ve only got two and a half billion active parameters, break bigger jobs into smaller pieces with explicit instructions. another key thing is the harness. So their docs recommend Pi over OpenCode They refer to reports of Pi being about one and a half times faster with workflows and I gotta say we’re really getting into the point now where, testing a model comes down to as much about the harness that you’re using as just the model itself All right, so let’s talk a

05:49 little bit about who’s behind it and why would a brand new lab ship a small local coding model as its very first product? All right, so the main public face of Recursive is Richard Socher and if you’ve worked in NLP at all, you probably know who he is. He used to basically teach a lot of the cool courses, at Stanford sort of 10 years ago he used to be the head of AI at Salesforce after they bought his startup. and for the past few years, he’s been working on a startup called You.com to try and take on Google at search Richard has always been one of the

06:21 people in deep learning who’s had really interesting ideas he was one of the people on the original ImageNet paper. He also did things like GloVe embeddings And the interesting thing is that even if you go back to his 2014 Stanford PhD thesis, it was titled “Recursive Deep Learning for NLP and Computer Vision.” so he’s had a lot of the ideas Around recursive self-improvement and this whole idea of sort of continual learning, getting models to be better, et cetera just recently he basically

06:52 published a book with a lot of these ideas called “The Eureka Machine.” In fact, I think it just came out in September. It’s something that I’ve been reading over the past few weeks, and like I said before, many things with Richard Socher, he has lots of good, interesting ideas now he pulled together a team of eight co-founders from a whole bunch of different places like Google, OpenAI, DeepMind, Meta, et cetera And there are a lot of seriously impressive people in there who’ve been working on a lot of original ideas for quite a while, not just on LLMs.

07:23 so you’ve got Tim Rocktäschel in there who came from Google DeepMind, where he was leading the Open-Endedness team, which was the team behind Genie, what you’ve probably seen was Google’s big world model. You give it a single image and even a rough sketch, and it can then turn it into a game. It can do a whole bunch of different things, work with it and in fact the entire team here is really well pedigreed, right? They’ve all been working on really interesting ideas and I think that kind of shows when in May they came out of stealth, having raised $650

07:55 million at a $4.65 billion valuation. The team is still only about 30 people or less. and you’ve got to think here that investors really weren’t sort of buying into any particular product. They were buying into a team and a system for how to basically make these models just improve themself, and be able to build that recursive self-improvement loop and that’s what brings us to the Eureka Machine. So the name gets used for two things. one being the big sort of vision from Socher’s book, and then two, the

08:27 actual system running inside Recursive. So let’s focus on the real system. So it’s a loop. An agent proposes an idea to try. It writes the code, it challenges itself, it runs a bunch of experiments, it trains something or benchmarks something. And then winners become sort of stepping stones for the next round and the interesting thing is that it runs lots of these ideas in parallel, right? So there’s lots of these research paths going on in parallel, and then it combines just the most promising ones And to be honest, this is kind of familiar territory to anyone that sort of looked

09:00 at the open-endedness research it’s a very similar sort of line of work, but scaled up with a lot more money, a lot more compute and the thing that they talk about which I think really does clarify a lot of things around this, is the hard part wasn’t coming up with the ideas, it was coming up with the judge And the challenge is as search got stronger, the candidates for judging sort of end up starting to be gaming the evaluation. So things like caching outputs, exploiting timing And the only way

09:30 you can sort of do well out of that is to then keep getting your evaluator to be getting stronger as well. Very similar to things like GANs, if you remember the old days of GANs, where you had a discriminator and a generator Socher himself has said that the team, have found lots of bugs in their own test harness when they were building this And I think there are lessons here that apply to anyone running agent loops or doing RL. your evaluator or your verifier really is kind of like an attack surface and the better that your optimization gets, the more holes it will actually find in

10:04 those kind of rewards so you can kind of think of this as like a really clever intern who will find a way to hit whatever metric you set whether or not that’s actually gonna lead to sort of the final goal Now, back in June, they were playing with this system on things like nanoGPT or nanochat, basically trying to get the loss down, there But Flint is really their first foray into actually making their own model which is a decent size and people can actually use, et cetera. One of the criticisms leveled here is that we still don’t know what’s actually running it.

10:36 which models, how many agents, how much compute, et cetera Are they, for example, just using a frontier model from one of the big companies to optimize their own models? Alright. So when we actually look specifically at Flint, Recursive says the system here sort of consisted of three main parts. So the first is architecture search. and it started with a model with thirty-five billion total, three billion active, Basically something very similar to the Qwen 3.6B model they then explored different expert setups and widths and landed on 40 billion

11:07 total with two and a half billion active So the fewer active parameters basically gives us the speed up here. And I will mention something that I’ve mentioned in other videos. it does seem to be kind of coincidence that the right size for your active parameters seems to be roughly around 5%, 6% of what your total parameters are and while that may change at really big sizes, MoE models that I’ve reviewed that have tended to do really well tend to stick to that formula, and ones where they go up to 10% active really don’t seem to do that well.

11:41 The second thing that they did, which was really kind of interesting, was the output layer. So the output layer obviously is gonna use a lot of parameters because you’re gonna be making a prediction where you softmax over all the possible tokens and interestingly here, Flint has gone for a tokenizer of about 150,000 tokens and just that output layer alone is around sort of 300 million, of those parameters. It works out to be roughly an eighth of everything that you’re gonna put each token through now most models won’t quantize that, but here they did quantize

12:13 that to four bits, and they actually recovered the accuracy by distilling from a 16-bit version of the model This is where they’re implementing that quantization-aware training This actually could be one of the key things that gets their NVFP4 version to score so close to The full precision model, which is kind of amazing there the third thing that they did was they actually sort of incorporated the whole idea of kernels in here. So on Apple Silicon, their system actually wrote a low-level Metal megakernel,

12:44 which sort of fuses the whole decode pass into one kernel and that just speeds up everything because there are fewer kernel launches, fewer trips to memory, et cetera now, it’s really interesting that they’re claiming that their Eureka Machine system actually sort of found these things and did it. unfortunately, there is no published paper, for this at the moment, so Really, we don’t have a lot to go on, but it would be really interesting to see that! Let’s jump into the demo and have a look at how the model itself actually performs Okay, so you can see when I come in here, I’m basically running the NVFP4 one.

13:17 I’ve run a bunch of the benchmarks on this myself. I definitely see, some differences, compared to, their official ones, both, positive and negative. but that also does sort of show how much, these things can actually, sort of change as you run them. if we come in here and just give it some sort of standard tests, you’ll see that we’re getting super fast, tokens out, and that’s because I’m running the NVFP4 version here, right? so you can see also just looking at the thinking in here, I’ll let that just run.

13:49 if we look at the thinking in here, it’s quite verbose in the thinking. it has gone through… In this case, it’s basically just sort of done, an evaluation out. It hasn’t actually done any sort of what I would call numbered thinking in here. we go through, it ran the task and it passed If we give it something a little bit harder We’ll see for a number of these that we’re not getting what I would call numbered thinking in here. if we look at the output, this is basically generating, pretty quickly. I’m generally seeing sort of like around the 200 tokens per second. If I give it something that’s more sort of freeform like this, let’s see

14:22 what the output, that we’re getting is And you can see here now this time I’ve basically given it a system prompt as well. we’re getting, quite a bit of thinking, going on here, and the thinking can vary quite a bit on this model. I’ve seen with these kind of things, sometimes the thinking is just for a few hundred, tokens. Sometimes it can be in the thousands. Now you’ll notice the other thing here is we’re getting the numbered thinking. So you can see here we’ve got the numbered thinking, going on. We’ve got like deconstruct the request, initial brainstorm, structuring the essay, drafting the content, really sort of a stock standard good, sort

14:58 of thinking process, around this. certainly not like the Neuralese that we’ve seen from some of the more modern models, where they tend to sort of abbreviate the thinking down But here you can see that we’ve generated, almost 6,000 tokens. We’re getting a bit over 210 tokens, per second, which is kind of around the average. I would say I’m sort of seeing around, anything sort of 200, 212. That’s quite common here. Now, you can see that there wasn’t a lot of pre-fill in this one. So what I’ll do is I’ll basically give it the same essay back and

15:29 then ask it to basically grade it. So we’ve actually got quite a lot of pre-fill going on Okay, so I’m asking it, “Please look at this essay and measure and check it for accuracy.” This is exactly the text that it wrote before. We can see coming out of it, we’ve got a fair bit of thinking in there, just going through that. and then we’ve got the overall assessment. it’s interesting that it only scores itself around 85% accurate. in this case, if we go through, it talks about different things that were… that it sort of wrote in the first one that are incorrect.

15:59 It then goes through and gives some revisions for it. You see this one we had pre-fill was over 2,000 tokens, and the token speed there for the pre-fill is super fast. And you can see the generation speed is almost exactly what we had, before And when we ask it to rewrite it, it’s basically then going through and doing that. if we look at sort of agent tasks, you’ll see that, it seems to be very good at using tools, from my tests. there’s no issues around things like that it’s able to go through different steps, run all those things, quite well And the main advantage of this is

16:33 just that it’s a lot faster, especially if you don’t have a very fast GPU. Now, the version I’m using here is definitely not as optimized as the one that they made the megakernel for on the Mac. that seems to be, a very unique thing that they’ve done. It would have been a nice thing to see them take that Eureka Machine and then point it at the, CUDA kernels and actually improve those as well. But I could imagine that that’s actually something that’s reasonably quite valuable They perhaps don’t want to give away the CUDA optimizations, Just like we’ve seen

17:03 with ThinkingCap, where a lot of these companies are giving away one version, but then charging enterprises for other versions, et cetera If you’re looking to see where this will fit on your GPU and perhaps, what you could actually do with this, I put this together basically just showing different GPUs, and, will you be able to run this or not? even on the RTX Pro with 96 gigabytes, I still can’t run the full bfloat, 16 version in there. You’re gonna probably need like an H200 or you’re gonna need multiple GPUs, to do this. you can run the NVFP4 version though quite, easily not to mention a bunch

17:36 of the, Unsloth quants and other quants that are out there, that you can run on much smaller GPUs Alright, so where does that leave us? Well, if you’ve got a pretty powerful, Mac Pro with 48 gigs of memory and you want a fast local, coding model, I think Flint is really well worth trying. if you’re on something like a DGX Spark or a bigger GPU, you’ll still get a fast mixture of experts coder. probably not gonna get that 130 tokens per second headline there Now for the Eureka Machine itself, We’re gonna have to see, do they scale up

18:06 to try and do a frontier level model? I’d love to see them take on a dense model like the 27B and actually beat that and then don’t forget we’ve got Qwen 4 just around the corner with a whole bunch of Engrams ideas and stuff like that, which they took from DeepSeek So I do think we’re gonna see a lot more ideas of this recursive self-improvement stuff, starting to take off. Anyway, let me know in the comments what you think. If you’ve tried out Flint, how was it? How has it held up for you? I know many of you are really interested in the MoEs rather than a dense model

18:37 just because it’s easier to run but it still does seem that you take quite a hit, when you do that as opposed to going for a fully dense model. anyway, as always, if you found the video useful, please click like and subscribe, and I will talk to you in the next video. Bye for now.