Do Diffusion LLMs Scale? Testing Mercury 3 + Decide

Author: Sam Witteveen · Watch on YouTube

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: Do Diffusion LLMs Scale? Testing Mercury 3 + Decide · Channel: Sam Witteveen · 21:28 · Watch on YouTube

Topics: AI agents, AI evaluation and benchmarks, Model inference and optimization, AI models, Multimodal models

Summary

The video examines whether diffusion language models can scale, using Inception’s Mercury 3 and Mercury Decide as examples. Mercury 3 is presented as a fast reasoning model that performs competitively on selected benchmarks, while Mercury Decide provides quick text- and image-based decisions with confidence scores. The host demonstrates their speed and capabilities in writing, tool calling, retrieval, image checks, and a support-ticket pipeline that combines both models. The conclusion is that diffusion models appear to scale and may be useful where low latency or high-volume decisions matter.

Key points

Unlike autoregressive models, which generate tokens sequentially, diffusion language models refine multiple masked tokens in parallel across several passes.
The video reports that Mercury 3 generates over 1,000 tokens per second and performs in a similar range to GPT-6 Luna and Claude Haiku 5.5 on some benchmarks.
Mercury 3 is positioned for reasoning, instruction following, search, coding sub-agents, and other workloads where latency matters.
The host says Inception has increased Mercury’s total parameters by more than 40 times from Mercury 1 Mini to Mercury 3, while its intelligence average rose threefold.
Mercury Decide is a decision model: it takes a state and questions, then returns answers and probabilities without generating a regular text response.
Mercury Decide supports both text and images, and the video highlights its use for classifications, visual checks, and multiple questions about one context.
The host demonstrates Mercury 3 on a long essay, agentic tool-calling tests, and retrieval tasks, and Mercury Decide on support tickets and image evaluations.
A combined workflow uses Mercury Decide for ticket triage, Mercury 3 to draft a reply, and Mercury Decide again to check the response.
The video suggests Mercury 3 for latency-sensitive applications and Mercury Decide for high-volume checks, particularly when confidence scores can help escalate uncertain cases.

Chapters

Start	Chapter	Summary
00:00	Diffusion LLMs and the promise of speed	The host introduces Mercury 3 and Mercury Decide and asks whether diffusion language models can combine speed with strong performance.
01:35	How diffusion language models work	The video contrasts sequential autoregressive generation with diffusion models that refine many tokens in parallel, then discusses the question of scaling.
03:49	Mercury 3 benchmarks, speed, and use cases	The host reviews Mercury’s reported scaling and benchmarks, its speed and pricing, and examples of latency-sensitive applications.
07:31	Mercury Decide and decision-model evaluations	The video explains Mercury Decide’s text and image inputs, reported evaluation results, fast judgments, and confidence scores.
09:40	Demonstrating Mercury 3	The host shows diffusion generation frame by frame and tests Mercury 3 on a long essay, tool calling, and retrieval.
14:56	Demonstrating Mercury Decide with text and images	The host tests rapid support-ticket classification and image-based judgments, including checks of generated clothing catalog images.
19:12	Combining the models and final takeaways	A workflow pairs Mercury Decide’s ticket triage and review with a Mercury 3 draft, before the host summarizes where each model fits.

Quotes

The diffusion on the other hand, works much more like an editor. — 01:35

Mercury 3 can get 1,068 tokens a second on NVIDIA Blackwell — 05:52

These models don’t generate any text at all. — 08:03

this is the advantage of diffusion, right? — 12:20

they show that diffusion does seem to scale for LLMs. — 21:28

Concepts and entities

diffusion language models, autoregressive models, decision model, Mercury 3, Mercury Decide, Inception, Jev, OpenRouter, Sam Witteveen

Full transcript (original language)

00:00 Okay, so most of the big AI labs are racing in one direction, and that’s that they’re fully focused on intelligence. Every new release is about topping some other benchmark And really when it comes to speed, that’s not their main concern at all And for a long time, that’s basically just been the story with autoregressive models, right? You’re predicting one token at a time. Well, diffusion language models have been the exception here. They’ve always been much quicker than normal LLMs. But for a long time, they came with a trade-off.

00:31 They just didn’t perform as well. You could kind of think of them as the fast little model that you could use for autocomplete, but not the model that you could hand a real agent task to Well, I’ve got to say with Mercury 3 and Mercury Decide, that’s all changed Mercury 3 itself runs at over 1,000 tokens a second. and on Inception’s benchmarks, it sits in the same range as GPT-6 Luna and Claude Haiku 5.5. So you’re not having to give up all your intelligence just to get speed anymore and Mercury 3 is not the only thing that they’ve been working on.

01:04 alongside that, there’s Mercury Decide, which is their take on a Jev-style decision model and as I’m recording this, it’s currently free on OpenRouter Now, when Inception reached out to me to give me access to this early model and sponsor the video I was really curious to see just how far diffusion models have actually come for LLMs So in this video, I’m gonna go through the two models. We’ll look at benchmarks, both some from inception, But also some third-party ones, and I’m gonna show you some tests that I’ve run myself And actually see where

01:35 diffusion language models are these days Okay, so if you don’t know anything about diffusion LLMs, let me explain a little bit about them. So how are they actually different? Well, most LLMs are autoregressive. you can kind of think of that as them writing like a typewriter. One token, then the next, then the next, and every single token costs a full pass through the model. That’s why decoding speed is so important. The diffusion on the other hand, works much more like an editor. They start with a rough masked out draft of the whole response and

02:07 then they refine lots of tokens in parallel over a handful of passes so each pass does more useful work. and because inference is limited mostly by memory bandwidth and not compute, getting lots of tokens out of each parse is where the speed up actually comes from and if the idea of diffusion sounds familiar, it should, right? It’s the same basic idea behind most of the image and video generators that are out there now I gotta point out that’s why it’s no coincidence that Inception’s CEO, Stefano Ermon, is one of the co-inventors of some of the key

02:42 diffusion methods behind modern image and video generation And the founding team at Inception have worked on techniques like this for quite a long time now, and many others that we use like DPO, flash attention, et cetera so this team is clearly a bunch of people who got diffusion early, And they’ve really been working hard for a while now to prove that this can work just as well for text, not just images And so the big question with diffusion LLMs has always been, do they scale? back at ICML in 2024, I attended Sander Dieleman’s diffusion circle

03:17 and specifically asked him at the time about this. Did it look like diffusion was gonna take over LLMs just like it had done with image generation and video generation and even things like, music generation And at the time then it really was an open question. No one had been really able to show that these things could actually scale up to sizes where they became very strong language models, now Inception are really the ones who’ve shown that this can scale And it’s pretty cool that a reasonably small startup is actually doing this. so back at the start, they started with Mercury 1.

03:49 They showed how you could build a commercial diffusion model and if you look at this chart, it really just shows what’s happened since. from Mercury 1 Mini to Mercury 3, they’ve scaled total parameters by more than 40X, and their intelligence average has gone up by 3X Now, the bit I find interesting is the jump from Mercury 2 to 2.5 was roughly the same size model. so it seems like that gain came from training, not scale. But then from 2.5 to 3, they’ve really scaled up to a much bigger model here.

04:20 so both levers seem to be working for diffusion Which is exactly what we wanna see if we’re gonna see this take off. Don’t forget we’re talking about a startup that doesn’t have anywhere near the funds of the big players like OpenAI and Anthropic, et cetera okay, so how does Mercury 3 compare to models in its class? this is against basically GPT-6 Luna, Claude Haiku 5.5, Where it’s strong is in reasoning, instruction following, and search. It’s doing very well on GPQA Diamond there Infact You can see that it’s

04:51 doing well on many of these benchmarks, including the agentic search benchmarks while it’s not at Opus 5.5 level for agentic coding, it’s still well-suited for medium-level tasks and as a default model for coding sub-agents And that whole idea of basically having like swarms of agents is really suited to this model. When you’ve got something so fast like this, you can deploy it for a whole bunch of different ideas like recursive self-improvement loops, understanding large code bases,

05:22 And anything where you’ve got a really large amount of data that you want multiple sub-agents to go across and interpret that data and if you’re building something like a search agent or a support bot where following instructions really matters and speed matters, Mercury 3 is looking really strong here now that’s just comparing the intelligence benchmarks. If we start to look at speed, then Inception’s numbers really take off. So Mercury 3 can get 1,068 tokens a second on NVIDIA Blackwell if you can compare

05:52 that to those other models, that’s about eight times what Luna gets, and even 4.5 what Claude Haiku is getting and as you’ll see in the demo that I show, this thing just flies at being able to generate text out And you also have the ability to turn the reasoning effort up and down and you’ll see just how quick that actually can make answers coming back Alright, so where does this actually matter? So Inception have a few early customers sharing numbers The one I like was OcTen, who do search on smart glasses Their search was taking about 80 milliseconds,

06:26 so the model was the bottleneck there. With Mercury 3, The generation time is 0.9 seconds. and you can imagine on smart glasses anything that’s gonna take just a few seconds is suddenly gonna make people think that something is broken there. Now the price for this at launch is gonna be 10 cents in and 35 cents out per million tokens, which is actually 50% off. So it’s gonna be interesting to see how long they run that. So longer term it may be a little bit more expensive than Luna or Haiku but you’ve still got the advantage of things like cached inputs being 90% off

06:58 And if your agent resends the same context every step, that’s going to really add up. so the idea is you’re paying for speed here, not saving on tokens Another example that they talked about was Tinyfish, who build web agents And in their blog post, they talk about other examples of people basically being able to do a lot more just because the model is so much faster and the cool thing here is that this is fully OpenAI compatible, so switching is basically just changing a base URL change. it supports tool calling, it supports structured outputs and they even support

07:31 a context window of a million tokens up from 260K in their last model And it’s great to see also that they’ve made this very easy for people to try out. Not only is it available on their own API, it’s on OpenRouter, Base10, and Vercel okay, so Mercury 3 is the thinking model. It’s the system two side of things. But as I’ve been talking about on the channel for quite a while, a lot of things you want a really quick system one kind of decision That’s the whole sort of thing, like agents just needing really quick calls.

08:03 Is this a billing ticket? Is an agent stuck? did a page actually load? And I’ve talked about it quite a lot on the channel, that that’s what decision models are for These models don’t generate any text at all. You give them some state and a question, and they give you back an answer and a probability in one call. so here’s where Mercury Decide fits in it’s a diffusion decision model you pass in a state and the cool thing about this is that can both be text but it can also be images as well. you pass in a set of typed questions And then the model will process all of that and return its answer in a

08:36 fraction of a second, so how good is it? so rather than just take Inception’s word for it, let’s look at some of the independent evals for this one. Val’s AI ran it through the same test they used on Jev on claim verifications, it was already matching Jev. On LegalBench, Mercury Decide is clearly ahead scoring around 87% versus 74%. And here’s the cool thing. It costs about a quarter of what Jev does And there’s a free version that you can try on OpenRouter as well On artificial analysis decision index, this model is third

09:10 on not hallucinating at 32%. And like I’ve talked about in the other videos about decision models, It really matters when you want a model that says it’s not sure instead of just guessing And then on speed, Valls measured a median of about 0.16 seconds for a single judgment and 0.4 seconds for 64 judgments so clearly a super use case with this is where you’re passing in one context and asking lots of questions about that at the same time now the new thing with the latest

09:40 release of Mercury Decide is vision and I really think that this is a pattern a lot more agents are going to use. Cheap System 1 checks wrapped around a single System 2 call you save the expensive thinking for the parts that need it, and you get a confidence score at every step that you can use to escalate to a human if you need it Okay, so jumping into the demo here. I’ve put together a little app, for testing this out. and I’m gonna show you a number of different things in here that I think are really cool with this testing suite and with my normal testing suite. So first off, if you’ve never actually used a diffusion LLM, you’re probably

10:13 gonna get a sort of shock at, one, how quick it is, but two, also just how it actually does the tokens. So let’s look at this. I’m gonna ask it to explain how diffusion language models differ from auto-regressive ones. So you’ll see that it’s gonna take a bit to kick off. I’ve got some settings over here. But you can see very quickly, look at it going, and you’ll notice also as it goes through, it’s already finished. okay. What we can do here is I’ve put in a feature to replay it and slow it back down. if I come in here and replay this, you can see I’ve got it step back,

10:47 so we can see what actually happened at each frame, as we go along here. And you can see that it gradually sort of fixes things up, as it goes through. It works out different tokens, like, okay, how the different tokens should come together, and it’s very different, than just sort of like a standard LLM, right? In fact, I can scroll through this manually, so we can just slow it down and look at it. Right? So with a normal autoregressive LLM, once it’s committed to a token, it

11:17 can’t really go back and change that. Now, there’ve been some papers that have tried different ways of backtracking and stuff like that, but fundamentally, it’s doing it step by step. Here you can see it starts to sort of plot out, like, a few different paragraphs, and then it goes through and actually sort of, refines this. So we can see that, as it’s going along, it’s working out, what should be at that start there. And so it had, when we go back here, it had at there, but it’s still–

11:47 and it’s got auto something SemIf. but as we go through the frames, it starts to basically put it all together. and sure enough, it then assembles, whole, sorts of paragraphs, and, eventually, gets through and gets right to the end, where it’s assembled the whole response here. And you’ll see that the response reads perfectly, right? There’s nothing, unusual about the text, that the Diffusion LLM makes. It’s just how it makes it, is fundamentally different.

12:20 So this is one of the first things that is really cool to actually see here is just the speed of how this happens. And you can just see there, that was so quick. Let me just run that again so you can see it. I’m not speeding anything up here. I’m just showing you the actual speed, right? It takes a little bit of time for the first tokens to come in, although it’s so quick I don’t even get a chance to say that. but fundamentally then it assembles it all, and it’s done very quickly in there. so this is the advantage of diffusion, right?

12:51 And you’ll see if I give it one of my standard prompts about writing a 5,000-word essay, this is, still gonna take it a little bit of time. It’s 5,000 words, and you can see the time to first token there is, taking a little bit to get going. But as it goes through, it basically puts all of this together, and gives this out. And if I speed up down here, we can see it’s still assembling, the end of the document. So it’s going through in chunks, putting it together. But this model is actually very good at being able to put the whole thing together.

13:23 So in my tests so far, I don’t find it doing the things that a lot of the open models do of where they only write half of the essay or, they prompt it to, “Oh, if I was gonna continue, I would get to this and this and this kind of thing.” so that’s one of the interesting things here And you can see if I come in and actually paste, that output in here, it’s 4,200 words, so that’s certainly a lot better than a lot of the models are producing out All right. If we come across to my normal sort of testing ground here, we can try

13:54 it out also on things like the, standard agentic tool calling stuff. And you’ll see that look, it’s just going through it all so quickly that, there are 10 tests in here, and often it would take quite a long time for an autoregressive model to do this. and I actually sort of pause it usually. I’m not getting a chance to pause it here. It’s gone through, got 10 out of 10, done really nicely on them. No sort of overthinking that we see on some of them in there If I come down and give it the sort of, pro ones, again, the speed of

14:26 the responses, is really good here. So, this is doing, really well, and this is one of the things that we saw, even some of the meta models having issues with, some of the things in here. But I’m gonna let that go through. it’s gone through and it’s basically passed all eight of them, perfectly, right? so this is one of the cool things about this model, right? That it can do all those kinds of things. And for things like RAG, it does a really nice job. You can see it’s gone through, very quickly doing, 4,000 tokens, going through

14:56 there to basically get the right answers. It was doing input to be able to extract things out So that one’s Mercury 3, which is their system two model, right? That’s basically where all the intelligence is, where it’s doing a lot of thinking, reasoning, that kind of thing If we change over to Mercury Decide, we’ve now got the System 1 model. and this is basically where you’ve got all your Jev-style decision model, things going on in here. So we can run things like a support ticket through this And you’ll see

15:29 the crazy thing here is literally I’m almost just pressing it, and it’s giving me the answer back, so quickly. so I will basically let it go, and you can see, okay, there it’s gone through. It’s basically done the classifications on this test. we’ve got the normals, we’ve got the choices, we’ve got, a score, in here. And the model’s done really well there. we can also see the confidence scores for things like choice and for things like, the score and stuff in there. if we want to see actually what got sent in there, we can see that’s just

16:01 a standard, API for a decision model. and then our raw response back is just giving us the probabilities, again, the standard for a decision model in here Now the cool thing about Mercury Decide is not only can we do all of that with text, right? We can also do it with images If I take a screenshot of this UI, and basically I’m setting it to actually do a UI screenshot, This is gonna be the screenshot of a page after a user has clicked Pay.

16:33 and then we’re gonna see, does it have an error? Does it look like the payment went through, et cetera? And you can see the response back there was super quick. and many ways, this is basically, showing, that’s our image there, which is actually just a screenshot of this UI. So really there’s not a lot related to payments or anything, like that there Right. If we try and do something like an image check, this is a really good, example I think of how people could use this. So what I’ve done here is I’ve used an image generator to make up some,

17:03 catalog pictures that would go on like an e-commerce site or something like that. Each of them is the same kind of outfit. One is with a male, one is with a female, one is with no human at all. And you can see the questions I’m asking in here. I’ve got a whole bunch of different questions in there. I’m asking it about, like, is there a human in there? Is it safe for work? a photo, a screenshot? what’s the quality? And stuff like that. Let’s just run it and see how it goes. And you can see already it’s done. I don’t even need to pause or anything with this.

17:34 So that was the photo, right? It’s a guy in an outfit there. Has a person, 100% yes. Safe for work, 99.8% yes. kind equals photo. person equals man. and then quality, it’s giving us 3.7 there. Let me swap out this image So now I’ve got basically the same outfit, but a woman in there. Let’s see how this one goes And you can see pretty much everything else stayed the same. the only thing that changed was a woman, right?

18:07 and that was quite deliberate. Like, we wanna have the same sort of look in here. We’re just changing one little thing in the actual image, but the model’s able to detect that and detect it really well Let me try a last one where we’ve got the outfit, right? The same outfit that those two people were wearing, but now there’s no human actually wearing this. Let me run it and you can see sure enough, it’s basically has person is 0%, still safe for work, in fact, slightly more safe, and then person, no human, in there.

18:41 so this ability to do these decisions really quickly really benefits from something like diffusion, right? You’ve got that, really fast response time. You’ve got the ability to ask lots of different questions and stuff in here Even in their benchmarks, when you ask sort of 64 questions, it’s not really slowing the whole process down, too much. It’s still extremely quick with its responses So just to wrap up the demo, the thing I would say is if you’ve never played with, diffusion language

19:12 models before, it’s a lot of fun. I gotta say it really is just fun to see how quick these things work. And I find it interesting how they’re putting their whole sort of ideas together, in a very different way than autoregressive models All right, so one cool thing that you can do is put these together. So this is an example of a pipeline that’s using both Mercury Decide and Mercury 3 in here. So you can see that, we’ve got some kind of ticket going in, and what it’s gonna do is basically do a bunch of classifications, go

19:45 through, ask a bunch of questions. Then it’s gonna use the system two model, Mercury 3, to actually draft a response. So let’s run this, and you’ll see that, it is actually spending the triage time of doing the decider stuff. Then it’s got the Mercury 3 running, and then it’s got more, checking at the end. So for this, it worked out that it was billing related. it worked out that it needed immediate attention. it needs a reply. it’s a refund request. And then the Mercury 3 came in with that information, was able to basically

20:20 draft a reply, very quickly, and then we can then check it again with the decision model just to make sure, did we do anything like overpromise? is there anything that’s sort of, not in there? What is the tone like? That kind of thing. So you can see that in this case, we’ve basically got it where we got a, a flat but acceptable to a bit of a warmer response back And in literally in a couple of seconds here, we’ve had the whole thing, be processed, checked, classified, all these sorts of things, in a really fast time

20:55 Alright, so where do these actually fit? Well, Mercury 3 makes most sense anywhere latency compounds. So agents, voice search, tutoring, all those kinds of things. this is gonna be a really useful model Mercury Decide on the other hand makes sense for high volume checks. So anything like moderation, routing, and especially anything where you can use the confidence score to escalate the uncertain cases Both models are currently on launch pricing right now, so definitely worth checking out And I think in closing, really what Inception has

21:28 done here is that they show that diffusion does seem to scale for LLMs. Anyway, let me know in the comments if you’ve actually tried out these diffusion language models. If you haven’t, I really suggest trying Mercury 3 out and just seeing how well and more importantly, how quickly it can actually work for your use cases as always, if you found the video useful, please click like and subscribe, and I will talk to you in the next video. Bye for now