NVIDIA Nemotron-3.5 Bolt-74B-A9B Locally: It's Not Over

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: NVIDIA Nemotron-3.5 Bolt-74B-A9B Locally: It’s Not Over · Channel: Fahd Mirza · 11:53 · Watch on YouTube

Topics: AI evaluation and benchmarks, Software engineering with AI, Model inference and optimization, Local and open-source models, Multimodal models

Summary

The video tests NVIDIA’s NeMo-3.5 Bolt-74B-A9B locally across coding, visual reasoning, translation, and scientific reasoning tasks. It fails to find a gestation-date bug in a real horse-stud management app and fails twice to produce a working animated kebab-skewer visual. It performs strongly on an image-based social reasoning task and a hand-drawn circuit analysis, but its translations are mixed and include hallucinations. The creator concludes that the model shows impressive reasoning for a 2-bit model, especially with images, but should not be trusted alone for coding or general production use.

Key points

The stud-book app used a 330-day gestation assumption, while the video states that the real average for a horse is about 340 days.
The model failed to identify the incorrect gestation assumption and made unrelated changes to the app.
The model failed to create a working animated doner kebab skewer, even after receiving a screenshot and being asked to fix its output.
In an image-based story about a missing laptop and friends’ beliefs, the model answered all nine questions correctly.
The model’s translations of a Turkish newspaper headline into more than 80 languages were mixed, and the creator observed hallucinations.
On a hand-drawn circuit task, the model correctly answered all seven questions and reasoned from the wiring rather than trusting incorrect labels.
The creator characterizes the model as strong at image reading and step-by-step reasoning, but weak at agentic and creative coding and multilingual tasks.
The model is described as a 74-billion-parameter vision-language model with 9 billion active parameters per token, quantized to 2-bit and designed to fit in 24–48 GB of VRAM.

Chapters

Start	Chapter	Summary
00:01	Testing the model on a real app bug	The creator introduces a horse-stud management app and tests whether the model can find its incorrect gestation-length assumption, but it misses the bug.
04:44	Creative coding test	The model is asked to create an animated kebab skewer in plain HTML canvas, but its initial output and attempted screenshot-based fix both fail.
07:16	Visual reasoning and translation	The model correctly answers nine questions about a laptop’s location and people’s beliefs, then gives mixed results translating a Turkish headline into many languages.
10:20	Circuit reasoning and overall verdict	The model correctly analyzes a hand-drawn circuit, leading the creator to praise its image reasoning while warning against relying on it alone for coding or production.

Quotes

The app assumes 330 days of gestation, but the real average for a horse is about 340. — 00:01

So first test, the model has failed. — 04:44

Nine out of nine correct against the answers as per prompt and image — 08:17

it is quite strong at reading images and step-by-step reasoning — 11:22

I wouldn’t really trust it on its own for coding and general production use. — 11:53

Concepts and entities

NVIDIA Nemotron-3.5 Bolt-74B-A9B, llama.cpp, Docker Compose, PostgreSQL, Flask, Redis, Vast Compute, Fahd Mirza, Bursa Hakimiyet

Full transcript (original language)

00:01 This is a stud book manager. A breeding operations dashboard for a horse stud farm. It tracks stallions, mares, and coverings. Predicts each mare’s falling date, and counts down the days so staff knows when to start a falling watch. Hidden in the back end is a one wrong number. The app assumes 330 days of gestation, but the real average for a horse is about 340.

00:33 Every due date is 10 days early, and this mare, who is still days away from falling, is a flagged bright red as overdue. In a real stud, falling watch staff, vets, and prep are all booked for the wrong dates. So, that is a huge cost and maybe a deal breaker for any farm out there. Also, this app is not a toy or just a showy app. This is a real full-stack app

01:03 with a Postgres database, a Flask API, and an engine X front end with some Redis running locally on my system with Docker Compose, as you can see here. I’m going to test this bug fixing with this new Nvidia’s NeMo Tron Bolt model, which is already being served on my local system with this llama.cpp command with dspark. And if you don’t know what dspark is or llama.cpp is, just search my channel. At

01:35 the moment, it is consuming this much VRAM, close to 30 GB of VRAM on this GPU. If you’re looking to rent a GPU on very good price, you can find the link to Vast Compute in video’s description. So, let me give this model a goal to go fix the bug after finding it without any hint, without any hand-holding. So, let me run this. It’s a reasoning model. While it runs, let me introduce

02:06 you to this model. So, as I said, this is NVIDIA’s NeMo-1 3.5 Bolt, a vision language model with 74 billion total parameters, but only 9 billion active per token. NVIDIA built it by compressing its bigger 120 billion parameter sibling, then quantizing the experts to 2-bit with quantization-aware training or QAT. The result is a model file of about 22 GB, designed to fit in 24 GB to 48 GB of

02:39 VRAM and run alongside other apps. It’s a Mamba-2 and Transformer hybrid with mixture of expert. It takes text, images, and videos with up to 256 K tokens of context on NVIDIA’s own table, and which I will show you quickly. This is a benchmark. It has beaten gpt-3.6 35 billion, NeMo-1 sibling on most benchmarks, specially terminal work, browsing, and

03:09 computer use. It also ships with a small drafter model for speculative decoding, which is what we are using here with llama.cpp. I will also quickly show you where it is making some prediction, and then it is discarding. There you go. That is what is happening. So, predictor and drafter are selecting few of the tokens. And as I said, please check out the channel if you are looking to learn more around drafter speculative decoding. By the way, this is Fahad Mirza, and I

03:40 really welcome you to the channel. If you want to support the channel, please become a member. Please also subscribe to this free weekly AI newsletter at fahadmirza.substack.com. Okay, so the model has come back with the response. And you know what? This result tells me that model has actually missed the real bug. It computed its expected values using 330 days. So it verified the wrong number and called Crimson Empress’s overdue status

04:12 correct. So if I go back to this screen, you see the horse Crimson Empress has minus five days left, which is wrong, which is a bug. But model has missed it. Even if you look at the reasoning, it was just sort of going in circles. It has tried its best, but I think it was not able to actually figure it out. And it changed three other things like group by a grand DB board. Even, you know, my dashboard loaded fine before the run. So

04:44 it has already started the docker, so I’m just going to do the hard refresh in front of you. There you go, you see? It is still there. It should say that five days left, not overdue. So first test, the model has failed. Let’s do another one. So in the next test, we are asking the model to build a doner kebab skewer rotating in front of a gas heater as one HTML file with no libraries. We are testing how well it turns a creative

05:14 brief into a realistic animated visual using only plain canvas code. I already have started it, so let’s wait for it to see what it produces. The model took fair bit of time, and it planned the draw order and checked for overlaps before writing, then reviewed its own code logic line by line. So the reasoning looks pretty good, and hand on heart, I haven’t even checked it myself yet what it has produced. So let me open it in front of you.

05:44 I’m just pasting the link and running it. So there you go. So, this is what it has created. If you look at the code, the code I’ll actually quickly show you. So, this is the index file. So, you see the code actually looks clean. And then you know, it just couldn’t do anything here, really. So, you know what I’m going to do? I’m just going to take a screenshot of this and I will give that screenshot as an

06:14 image to the model because it’s a vision model and then we’ll tell it that it needs to fix it. Let’s do that. So, this is the screenshot, as you can see. I’m going to give it as an image to this model in the same session. And then I’m going to give it this prompt telling it that this is a screenshot and it needs to compare it with original brief and then go fix it. So, this is the last chance for the model, so let’s wait. Okay, second time the model says that it

06:46 has found the bug and fixed it. Let’s check it out because again, you know, the reasoning looks good. But did it actually fix it or what happened? There you go. No, nothing happened there. I’ll even do the controller file just to say you know, hard refresh, but nothing really happens. So, even the second code generation test has failed. Let’s do third one. For the next test, let’s go to Accra,

07:16 Ghana. You can see that this is a scene of a coffee shop on Oxford Street. Nella’s laptop disappears from the table, then moves again, and by the end of the story, three different friends each believe it is somewhere else. I’m going to hand the model this photo of the table and then I am going to ask it this prompt, which is let me show you the prompt. Sorry, there you go. This is a prompt which is asking a lot of questions that

07:46 who think it, you know, took the laptop to see if it can tell where the laptop really is from where everyone believe it is. So, you can just read through this prompt. Let me run it and see how exactly model does the vision reasoning. Let’s wait for it to come back. Should be fun. And by the way, all of these images which I’m using, not this one, which I showed you, all AI generated. Okay, at last, there is something to

08:17 celebrate for Nemo Tron. Nine out of nine correct against the answers as per prompt and image including the hard one like Coffee’s backpack, Aqua’s and Coffee’s beliefs about each other and the silver laptop being Coffee. So, each answer following follows the right chain of who saw what. A really good answer by Nemo Tron. And you know what? I have been covering Nemo Tron model from day one and I have seen really good experiences

08:47 especially in coding, too. I’m not sure what happened there with the bolt. Anyway, let’s do another test. For the next test, let’s go to Turkey or Türkiye. The image is a screenshot of a news article from a Turkish newspaper Bursa Hakimiyet. I know I’m not pronouncing it right and I’ll actually show you live. So, this is a newspaper where I have taken this. So, huge shout-out and credit to them. In this image, you see that there are some headlines

09:18 and few other things. I’m asking the model to go and then translate only the headline into each of the following languages. I’ll also show you that. There you go. So, there are 80 plus languages which it is doing at the moment. Once it finishes, please help me out. Check your language. Let me know what do you think about the translation of that headline. But you can already see that it is hallucinating. So, I think it is filling apart on this thigh.

09:50 So, which is not good, but it has done few others. So, these are the ones I’ll just quickly show you and stop it because it is going into the loop now. Okay, even sorry. Because I just want to show you in real time without editing. There you go. So, these are few of the ones. Let me know what do you think. The ones which I know, I think it it is quite an average response, but I will also take your advice. Some of the major languages are correct,

10:20 but overall, it is not uh something to write home about proudly. Major languages are good, but you know, in 2026, that is a given. Anyway, let me know your thoughts on this. Let’s now do a scientific reasoning test. For this one, what I’m doing, I’m giving it a hand-drawn circuit with a wrong label, a dangling wire, and a meter reading 0 V or 0 V with no leads drawn. So, what we are

10:51 testing here with this prompt is to see if the model reads the real wiring instead of trusting the annotations and then gets the numbers right. Okay, again, the scientific reasoning looks pretty solid. All the seven answers which it has given with every number are correct. I’m not going to read those numbers, but I’ll just scroll through so you can see and calculate. Feel free to pause it. Um it has also read the drawing correctly

11:22 instead of trusting the labels, where it is calling R1 R2 wrong and treating the dangling wire as carrying no current. So, look, overall, it is quite strong at reading images and step-by-step reasoning where, you know, a couple of the answers were really world-class, but it is quite weak at, I would say, open-ended, agentic, and creative coding, multilinguality because you already saw the results. But again, for a 2-bit model in this

11:53 contest format, the reasoning is impressive, but I wouldn’t really trust it on its own for coding and general production use. Let me know what do you think. Thank you for all the support.