MiniCPM-V 4.7 1B: Install with Ollama and llama.cpp, Then Push It to the Limit

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: MiniCPM-V 4.7 1B: Install with Ollama and llama.cpp, Then Push It to the Limit · Channel: Fahd Mirza · 09:24 · Watch on YouTube

Topics: AI evaluation and benchmarks, Local and open-source models, Multimodal models, RAG, search and information retrieval

Summary

The video installs MiniCPM-V 4.7’s 1B GGUF model using Ollama and llama.cpp, then tests its visual, coding, and reasoning abilities. The presenter reports that it uses just over 5 GB of VRAM and performs well on several image tasks, including reading a clock, generating a working clock from an image, and solving a math question. However, it struggles with handwritten text translation and misses parts of a complex creative prompt. The presenter concludes that it may suit lightweight image captioning, simple OCR, and on-device coding, but not complex reasoning or long multi-step instructions.

Key points

The 1B model can be installed using either Ollama or llama.cpp; the presenter recommends llama.cpp.
The model uses just over 5 GB of VRAM in the demonstrated setup, with the possibility of reducing usage by lowering the KV cache share.
The presenter describes the model as designed for edge devices such as phones and laptops, with coarse and fine visual modes.
It correctly reads an analog clock and creates a working clock based on an image.
When asked to identify a soldier’s country from an image, it avoids guessing and says it needs more context.
It identifies a dress as a traditional Indian-style lehenga or ensemble, though the presenter questions its assessment of the girl.
It fails to reliably extract a specific handwritten line and translate it into many languages.
It correctly solves a school math question using the vertex formula, despite a minor slip in its explanation.
On a multi-part creative image prompt, it answers only some tasks and misreads aspects of the scene.
The presenter suggests it for lightweight image captions, simple OCR, and simple offline coding, but not complex reasoning or long multi-step tasks.

Chapters

Start	Chapter	Summary

Quotes

If you want to do it through llama.cpp, which I would highly recommend, all you need to do is to use this command. — 00:31

Which is absolutely correct, and you can see that how fast this model is. — 03:43

So, the 1 billion cannot read this handwriting reliably or translate beyond a few major languages. — 06:50

That’s it. Let me know your thoughts about this model. — 09:24

Concepts and entities

MiniCPM-V 4.7 1B, MiniCPM-V 4.7 35B A3B, Ollama, llama.cpp, GGUF, VRAM, KV cache, OpenBMB, Fahd Mirza

Full transcript (original language)

00:01 Open BMB is on the road. Yesterday, we tested the big mini CPM V4.7, the 35 billion A3B version. Today, we look at its little sibling, the 1 billion version, a tiny model that reads images, multiple images, and video. We are going to install the GGUF with both Ollama and llama.cpp, then test it. This is Fahad Mirza, and I welcome you to the channel. Let me take you straight

00:31 to my terminal where I have this Ubuntu with this 48GB of VRAM GPU card. You don’t really need that much GPU, and I will show you the VRAM consumption. If you want to use Ollama, simply run this command. You can also use the bigger sibling with it, too. You can see how small this model is. So, this is Ollama-based model. If you want to do it through llama.cpp, which I would highly recommend, all you need to do is to use this command.

01:04 And it is going to download this model. By the way, if you’re looking to rent a GPU or VM on very good price, you can find the link to Mass Compute in video’s description. And please subscribe to this free weekly AI newsletter at fahadmirza.substack.com. While it serves, a few lines about this model. This 1 billion version is built to be small without feeling small. As you can see on this chart, which is from their own blog, the closer

01:35 a model sits to the top left, the better. By the way, the big 35 billion star sits well above its current rivals, scoring higher while finishing faster. For the 1 billion, the star sits near the top left, well ahead of the Quen’s 0.8 billion model, which are slower and score lower. And it keeps pace with Quen’s 2 billion model in instruct mode at the fraction of the time. As I said in my last video, it feels that Open BNB is trying to be next Quen.

02:08 How much they have succeeded or succeeding, I will let you be the judge. But I think if they keep doing what they are doing, they might not be that far, especially after you check out this other 35 billion parameter model. So, as I said, this model is being built to be small without feeling small. It understands single images, even multiple images, and it squeezes visual detail early inside the vision encoder. So, reading a picture car costs far less

02:40 compute. You can even switch between a course and a fine visual mode depending on whether you want speed or detail. So, I’m going to also test it out on both course and fine shortly. It is designed for edge devices, phones, laptops, and chips. And there are various quantized versions if you want to go even with higher precision. Meanwhile, our model is being served, and this is the VRAM consumption. Just

03:11 touch over 5 gig of VRAM. You can reduce the KV cache share in order to even bring that VRAM consumption lower. Okay, for the first test, we are going to give it this clock image, and I’m testing if this 1 billion parameter model can read an analog clock, which means telling the hour, minute, and second hands apart. It has already Okay, so it says that I can see the clock shows approximately 10:10, and then it is breaking it down. You see? This is

03:43 the answer. Which is absolutely correct, and you can see that how fast this model is. Next, let’s proceed with a coding task. I’m asking it to look at this image and then simulate this a working clock. Let’s see if it is able to do that. And we can just continue here. And there you go. This is what a 1 billion model in quantized format has

04:14 created. This is very impressive. And this was the image if you’re looking for the similarities. The vision plus coding is quite impressive. Okay, let’s check out another one. This is an AI-generated image. And I am asking the model which country this soldier could be from. Let’s see if it is able to deduce that or not. And there you go. I think this is quite a good response because model was unable

04:45 to determine that. It is quite a fair. But instead of hallucinating and giving us a wrong answer, it is admitting that it was not able to do so because it would need additional context. Let’s do another one. In this one, I’m asking it what would be the origin of this dress and the girl. Let’s see because they seems to be from two different regions. And this is where it has identified that

05:16 this could be the Indian style lehenga or traditional Indian ensemble, which is correct. It could be from Pakistan, too, by the way. It is quite rich embroidered. But for the girl, I don’t think so this one is correct. As you can see. But you can be the judge, too. I think the first part it got it right. Now, in this next test, I’m giving this model a photo of a handwritten note and ask it to read one specific line, the

05:49 second to last, which is writers in love. And then there are some crossed out words, too. And the prompt is quite hard. You can see that I’m asking it to not only pick up that second last line, but also translate it into these almost 80 plus languages. Let’s see what model does. Meanwhile, let’s have a quick look at this benchmark table, which puts this model up against its predecessor and other small vision models. It leads on

06:20 general vision, reading text and images, chart reasoning, and multi-image test, and beats the smaller coin and LLM models in most rows. It trails on math and a few document test, and the cheaper 16x time mode stays very close to the finer 4x mode. And there you go. For this multilinguality test, it has just um fell flat on its face. You can see it misread the line,

06:50 merging several lines of the note into one garbled sentence, instead of single line. That is the first failure. And then most of the languages are just English sentences copied or lightly mangled, including various low-resource, high-resource languages. So, the 1 billion cannot read this handwriting reliably or translate beyond a few major languages. And you know what? Even the multilinguality was not that good in its larger sibling.

07:21 So, let’s try to do another one. You can see that I’m going to give a photo of a school math paper and ask it to find and solve one specific question, which tests reading math notation plus step-by-step reasoning. So, let me quickly give that prompt, and let’s see if it is able to solve it or not. There you go, it has given me the answer. And despite of what Benchmark says, it has got it perfectly right. It used the

07:51 vertex formula and then picked the option B avoiding the 41 16 trap. So, there is one small slip there around option C, but I think that didn’t affect the answer. So, this is a full pass. Finally, let’s use this image to check out model’s creativity and see if it can do the visual understanding specially around expressions. So, this is again again an AI generated image and I’m asking the model that in two

08:23 sentences describe what is happening. Invent the funny thing that happened 1 minute before this moment. Give each of the four people a one-line thought bubble that matches their body language. Write a funny newspaper headline for the scene. Let’s see how model deals with this one. And the model seems in a hurry. It only answered the first of the four tasks giving the backstory, thought bubbles,

08:54 and headline. It also misread the scene saying everyone watched intently when the man was looking down and it missed the laptop. So, look, I think there’s a severe loss loss of performance too, especially when it comes to creativity. With coding and math, it is not bad at all. So, I think for lighter jobs like quick image captions, maybe simple OCR, and um simple coding on device or offline, this could be a good choice, but not for

09:24 complex reasoning or even a decent enough reasoning or long multi-step instructions. That’s it. Let me know your thoughts about this model. If you want to help out the channel, please become a member. Thank you for all the support.