This Tiny Coding Model Shouldn’t Be This Good (MiniCPM5)

Author: Better Stack · Watch on YouTube

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: This Tiny Coding Model Shouldn’t Be This Good (MiniCPM5) · Channel: Better Stack · 07:22 · Watch on YouTube

Topics: AI agents, AI evaluation and benchmarks, Model inference and optimization, Local and open-source models

Summary

MiniCPM5 is a small, llama-compatible model that performs remarkably well on some coding and agent benchmarks, even outperforming larger models in selected tests. Its agent-focused training and broad runtime compatibility make it easy to try locally, but its benchmark picture is mixed and the results depend heavily on sampler settings. The video highlights that quantized versions can loop badly with documented defaults, while changing the repeat penalty and minimum sampling value substantially improves performance. The presenter sees it as an interesting local model to experiment with, but would still choose Qwen 3.5 4B for larger tasks.

Key points

MiniCPM5 is presented as a 2.5-billion-parameter model trained with 500,000 agent trajectories, reinforcement learning, and 16 merged RL expert models.
The video reports a SWE-bench Verified score of 46, compared with lower scores for other small models and 34 for Qwen 3.5 4B.
Its llama-style architecture supports compatibility with runtimes such as MLX and llama.cpp.
The model can produce structured tool calls when run through llama.cpp and an OpenAI-compatible client.
Benchmark results are mixed: MiniCPM5 wins on some coding tests but loses to Qwen 3.5 4B on SWE-bench Pro, Terminal Bench, and several other evaluations.
Quantized GGUF versions reportedly loop frequently with default settings; setting the repeat penalty to 1.15 greatly improves the reported HumanEval+ scores.
The video says llama.cpp’s default min_p value of 0.5 is a problem and should be set to zero, though this was missing from the docs at the time discussed.
The presenter notes limitations in evaluation transparency, model-library availability, documentation, and the lack of measurements with thinking disabled.
The presenter recommends trying MLX 4-bit or Q8 GGUF with adjusted sampling settings, while preferring Qwen 3.5 4B for large tasks.

Chapters

Start	Chapter	Summary
00:00	A tiny model with a surprising claim	The video introduces MiniCPM5 and asks whether a small model can work well as a coding agent, while warning that quantized settings can cause looping.
01:37	Running MiniCPM5 locally	The presenter demonstrates running the model with MLX and llama.cpp, including chat and structured tool calls.
03:12	Compatibility and mixed benchmarks	The video explains the model’s llama-compatible architecture, then compares its strong results on some benchmarks with weaker results on others.
04:44	Quantization, sampler settings, and caveats	The presenter describes severe repetition in quantized runs, the improvement from changing sampling settings, and limitations in the available evaluations and documentation.
06:20	Who should use it?	The presenter recommends experimenting with adjusted local settings but prefers Qwen 3.5 4B for larger tasks.

Quotes

the broken version is fixable with one flag. — 00:00

the training was built around agents. — 01:06

If your runtime can load llama, there’s a good chance it can load this. — 03:12

The sampler isn’t just tuning anymore. It’s part of the actual model. — 05:17

The defaults are what can break it. — 07:22

Concepts and entities

MiniCPM-5, OpenBMB, SWE-bench Verified, MLX, llama.cpp, Qwen3.5 4B, GGUF

Full transcript (original language)

00:00 This two billion parameter model claims to beat the four billion one on S swb bench but run the same model at 4bit using the exact settings in its own docs and it can fall into an infinite loop over 90% of the time. Now that could be true but it’s also true the broken version is fixable with one flag. This is mini CPM52B from open BMBB. Is a 2 billion parameter model good enough to actually be used as an agent? Let’s take a look.

00:32 Now, small models have had the same problem for a while. They are getting pretty good at chat. Give them tools, though, and things usually fall apart. Look at swbench verified. These are real GitHub issues when the model has to produce real patches. Quinn 3.5, 2B scores five. Gemma 4, E2B scores two. Those are vendor numbers, but match what anyone who’s tried small models for agent work has probably seen. The model hallucinates the toll schema or forgets it or just loops. Then mini CPM 52B

01:06 shows up with a score of 46. Not only way above the other 2B models, but also above Quen 3.54B, which is twice its size. So, what’s changed? OpenBM points to three things. The architecture is plain llama. There’s nothing exotic here. It has about 128,000 native context and most importantly, the training was built around agents. 500,000 agent trajectories during supervised fine-tuning, then reinforcement learning, then 16 separate

01:37 RL expert models merged into one. It’s Apache 2.0 with open data sets. And just to avoid one easy point of confusion, this is the texton mini CPM5 model. Now, let’s actually run it. If you enjoy coding tools that speed up your workflow, be sure to subscribe. We have videos coming out all the time. All right. Now, for MLX, I’m on a Mac here. We can just do a pip install in a virtual environment, which I got set up here. Older versions of MLX LM ignore this model’s end of sequence list. So,

02:07 the model literally doesn’t know when to stop talking. Then, we can run the first prompt to pull the model in the weights. Just give it a second. It fires up the first time. And here it goes. Once we run this, you get the think block. This is how it thinks. Then after this, it produces that final answer. And there’s the tokens per second counter as well. All of that is coming from a 1.3 GB file. Now, we can even set this to be an interactive chat, too. So, when I drop this line in here, boom, it fires

02:39 up. You can see now it’s always listening. So, we can just ask a question on the go, and the responses that we get fire right back at us. Now, chat is cool, but the tools are just as good here. I’m going to start up Llama CPP now in a new terminal, then point an OpenAI client at it and give it a simple weather tool. The model produces a structured function call llama CPP parses it, and the response comes back as a proper tool calls array. So now you can think of all the different ways this could run cleanly on your system. Now,

03:12 there’s also a reason compatibility is this clean. If we take a step back, this is a dense 2.5 billion parameter model with 42 layers and grouped query attention. But the important line is the architecture here. Llama for casual LM. Mini CPM4 isn’t that. It used custom sparse attention and multi-token prediction. That’s nice and all, but every runtime needed special support. Mini CPM 5 basically throws that away. If your runtime can load llama, there’s a good chance it can load this. MLX,

03:43 Llama CPP, Web, LLM. So, compatibility is one of the strongest parts of this model. But now we get to the whole benchmark table because this is where it gets a lot less clean. If you only read the table one direction, Min CPM5 looks ridiculous. Against Quen 3.54B, a model twice its size. Live codebench has them 69 to 56. SWE Bench verified 46 to 34. You can check these rankings on other platforms as well. So, the comparison here is kind of ridiculous. That sounds

04:14 like a 2B model replacing a 4B model. Now, look at the table in a new light. SWEBench Pro 14 versus 28. Mini CPM 5 gets half the score. Terminal Bench, we have about an 8 1/2 versus around a 26, about a third. Quen 3.54B also wins on other platforms. Mmlu Pro, GPQA Diamond, and Logbench V2. and on plain instruction following even Quen 3.52B beats mini CPM 5. So when they call this

04:44 4B class there’s some truth to it for code tool calls math and parts of the agent workload absolutely across the whole board. Yeah. No, probably not. The bigger problem with this model specifically is what happens when you run the quantized model using the documented settings. Someone on the GitHub tracker tested human eval plus against the GGUF files. Q8 scored about a 43 and 55% of generations ran away into repetition. Q4KM scored around a six and 92% ran away. 92%.

05:17 That looks like a broken model except then they changed one thing. Repeat penalty 1.15. Q8 jumps from around a 43 to a 92. Q4 jumps 6 to 71. Open BMBB later said on Hugging Face that Llama’s CPP’s default min value is 05. That’s the real problem and that it should be set to zero. But that wasn’t in the docs when people ran this. At 2 billion parameters, one bad default can turn a useful model into well nothing really great. The sampler

05:48 isn’t just tuning anymore. It’s part of the actual model. And there are a few more things you should know before using it. Every benchmark here is vendor ran. So the comparison table only includes Quen and Gemma. No Llama, no fee, no small LM3, not even a mini CPM4 row showing what improved generation to generation. Someone asked for the evaluation scripts in an issue. No response. It’s also not in the Lama library, so you need to write a model file. There’s no LM Studio page either.

06:20 You drop the GGUF inn manually. Thinking is enabled by default, so every answer has to reason before it starts giving you the response. There is an enable thinking switch in the chat template, but the official docs only provide a sampling profile for think mode. Nobody has measured what it’s like with thinking disabled yet. English and Chinese only, more than 100,000 downloads for the 2B model, 600,000 for the 1B. People are running these, they’re just not talking about them that much. So, would I actually use mini

06:51 CPM52B? Well, these are really cool to play around with. They’re cool to see where local stuff is going for a local coding agent on maybe like a MacBook Air. I mean, yeah, okay. It could be kind of cool. I’d run the MLX4 bit or the Q8 GGUF. I’d set them in P to zero. Set the repeat penalty to what they actually recommend for that job. It might be pretty strong for that size, right? we can see where these local models are going. Give them a test and see how they improve over time. For anything that’s like a really big task,

07:22 I would still take Quen 3.54B over this. And that’s not me disagreeing with OpenBM. I’ve tested other models they have. Some are really, really good. The interesting thing here isn’t really that a 2B model can beat a 4B model. It’s how close to the edge these small models are actually getting. Mini CPM 5 itself is fine. The defaults are what can break it. I’m Josh from Better Stack. If you enjoy coding tips and tricks like this, be sure to subscribe. We’ll see you in another video.