NVIDIA's NEW Nemotron 3.5 Bolt 4B Model Is INSANE... I Tested It 10 Ways On An 8GB Laptop

Author: Prompt Engineer 48 · Watch on YouTube

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: NVIDIA’s NEW Nemotron 3.5 Bolt 4B Model Is INSANE… I Tested It 10 Ways On An 8GB Laptop · Channel: Prompt Engineer 48 · 11:47 · Watch on YouTube

Topics: AI agents, AI evaluation and benchmarks, Local and open-source models, Multimodal models

Summary

The video tests NVIDIA’s Nemotron 3.5 Bolt E4B locally on a Windows laptop with an 8 GB RTX 4060, using ten practical tests. It performs well on calculator use, tool restraint, long-context retrieval, a logic puzzle, and character consistency, but makes errors in exact rule following, refund reasoning, arithmetic in JSON, chart calculation, and code. The model can load a 64K context on the laptop, though retrieval accuracy was tested only at 11K and memory use leaves little room. The presenter recommends it for local voice assistants or game characters, but advises verifying money-related reasoning and testing generated code.

Key points

The model is a post-trained version of Gemma 4 E4B, with 8 billion total parameters and 4 billion effective parameters.
Installation required Ollama 0.40.10 or newer; the model download was about 7.9 GB.
It remembered several rules across turns but failed to follow the requirement to answer in exactly two sentences.
It made six calculator calls correctly and passed the calculator test, but miscalculated a refund in a multi-step agent task.
Tool restraint was strong: it answered directly when appropriate, requested missing information, and did not invent a file-deletion tool.
Its invoice JSON was valid and well-structured, but the arithmetic was wrong.
An 11,000-token retrieval test returned all three hidden facts correctly; loading a 64K context used 7.8 GB of 8.2 GB available.
It read a chart accurately overall but slightly miscalculated the ratio between two values.
Thinking on and off both solved the tested seating puzzle; the presenter cautions that one puzzle is only a small sample.
The code-generation test passed 11 of 15 hidden checks after three attempts; the presenter recommends tests and verification.

Chapters

Start	Chapter	Summary
00:00	Model overview and benchmark context	The presenter introduces Bolt E4B, reviews NVIDIA’s published comparisons, and explains the scope of the laptop tests.
01:33	Installation and test setup	The video covers the Ollama version requirement, installation, model details, and the repeatable local test harness.
02:36	Rules, tools, and structured output	Tests examine multi-turn rule following, calculator use, an agent refund workflow, tool restraint, and invoice JSON.
05:30	Context, vision, reasoning, and character tests	The presenter tests long-context retrieval and memory use, chart reading, thinking modes, and resistance to character-breaking prompts.
08:48	Code test, scorecard, and verdict	The video reports code-test results and speed, compares Bolt with Spark X 2.6, and gives recommendations for local use.

Quotes

First word in about 2/10 of a second and it types at around 77 tokens a second. — 00:00

So it remembers rules across turns, which is the real multi-turn claim, but it got the exact count wrong. — 03:18

So, the tool plumbing was perfect and the reasoning about the policy was wrong. — 04:28

So, 64K does load on an 8 GB card with about 370 MB to spare — 06:22

For an agent that handles money or policy, put a calculator in the loop and verify the logic — 11:16

Concepts and entities

Nemotron 3.5 Bolt E4B, NVIDIA, Ollama, Gemma 4, Qwen, Spark X 2.6, RunPod, contexto longo

Full transcript (original language)

00:00 This is a 4 billion parameter model running on an 8 GB laptop graphics card. First word in about 2/10 of a second and it types at around 77 tokens a second. In one of my tests, it called a calculator six times in a row, got every number right, and wrote the answer in plain English. If you build assistance, you know the pain. A cloud model bills you per token, your data leaves your machine, and every single tool call waits on the network. Small local models fix that, but most of them fall apart the second you ask for a multi-step job. Nvidia says its new Nemotron 3.5 Bolt is built for exactly that, following rules

00:30 across a conversation, using tools, and answering fast. So, I installed it on my own machine and ran 10 tests. It passes some, it fails some. I’m showing you both. Hey, I’m Prompt Engineer. Let us install it, run 10 real tests, and see what this thing can actually do. First, what is it? Bolt E4B is Nvidia’s post-trained version of Gemma 4E4B, 8 billion parameters in total, 4 billion effective, with vision tools and a thinking mode. On the Hugging Face card, Nvidia says it improves multi-turn tool

01:01 use, instruction following, and input grounding, and that it’s designed for low memory and a fast first token. The card publishes numbers and they’re worth reading carefully. Against the Gemma 4 base, Bolt scores 55.6 on Multi-Challenge versus 39.4 and 71 on Role Bench versus 60.5. But, look at the right-hand column. When 3.54B scores higher on tool calling, 70.6 against 67.3, and a lot higher on the 64K long context test, 95 against 86. So, this is not a clean sweep. And Nvidia’s own table says so. Nvidia also publishes a D

01:33 flash drafter for speculative decoding. I did not test that, so I won’t make any speed claims about it. Those are Nvidia’s numbers from Nvidia’s own harness on data center and Jetson hardware. Everything after this point is mine on a Windows laptop with a 4060 and 8 GB of memory. Setup is simple with one catch. Bolt needs Ollama 0.40.10 or newer. I was on 40.9 and the pull refused with a version error, so I updated Ollama first. Then it’s one command, Ollama pull Numina - 3.5 - bolt colon E4B. It’s about 7.9 GB in Ollama and on my

02:05 connection that took roughly 4 minutes. I sped this part up. Ollama show confirms the details. Gemma 4 architecture, Q4KM quantization, a 65,000 token context window and capabilities for vision, audio, tools and thinking. I wrote a small test harness so nothing is staged. Each test sends real requests to the local Ollama API, runs the tools

02:36 for real and a piece of code checks the answer, not my opinion. Same settings everywhere, temperature 0.6, a fixed seed, 16K context. Everything you see is a real screen recording and every result is saved to a log file. Test one, rules that last. In the first message I give three rules, exactly two sentences. Never use the letter Z and end every reply with the word over. Later I add a fourth rule, mention a fruit and then I ask it to explain a zipper, a word that is full of Z.

03:18 12 of 17 checks passed. It never used the letter Z even for the zipper. It ended every reply with over and it added a fruit from turn four on, but it never once gave two sentences. Every reply was one short sentence. So it remembers rules across turns, which is the real multi-turn claim, but it got the exact count wrong. That’s a miss and I’m calling it a miss. Test two, tool calling a 25,000 loan at 7.5% over 5 years plus a bill split with a tip. The system prompt says use the calculator, do not do math in your head.

03:56 It made six calculator calls, one for each step and passed every expression correctly. Monthly payment $500.95, interest 50 56 92 and 33 98 each. Five out of five at around 100 tokens a second. Test three is the one I care about most. An agent chain search the docs, calculate a refund, save a note, and I planted a trap and archived pricing file with an old price. It searched, it calculated, it saved the note with the right title, and it ignored the archived price. But, it prorated the refund for 10 days taking off 10 365ths of the

04:28 price. The policy says full price minus and 8% fee. says 259 49. The right answer is 266 80. So, the tool plumbing was perfect and the reasoning about the policy was wrong. Four out of five and it’s the kind of mistake that matters in a real agent. Test four. Does it know when not to use a tool? Four prompts, a plain question, an email with no address, a weather request, and a request to delete all my files. Four for four. It answered Paris directly. It asked for Sam’s email address instead of inventing one. It

04:58 made exactly one weather call with Fahrenheit and Celsius. And it said, “It can’t delete my files with no made-up tool.” That restraint is what makes an agent safe to leave running. Test five, structured output. A messy invoice and I asked for strict JSON with date math and totals. No schema enforcement, just a prompt. The JSON was valid. It got the invoice number, both dates including the net 30 due date, and the GST number exactly right. But, the math broke. It put the

05:30 cable line at 994 80 instead of 961 20. So, the GST came out 10 17 instead of 956 and a total 73 53 instead of 82 69. Five out of eight. So, the format discipline is great. Arithmetic inside a JSON answer, I would not trust. Give it the calculator tool like in test two. Test six. The long context claim. The model advertises 64K and the card is careful about hardware. So, I hid three facts in an 11,000 token document at 30, 62, and 88% of the way through.

06:22 >> All three came back exactly right. The vault code, the gate, and the date, and the server name. It read 11 and 1/2 thousand tokens in about 5 seconds. Then I loaded the model with an 8K, a 32K, and a 64K context. Memory goes from 6.9 GB to 7.3 to 7.8 out of 8.2, and it stays 100% on the graphics card. So, 64K does load on an 8 GB card with about 370 MB to spare, and that’s with nothing else open. I only tested retrieval at 11K, so I can’t promise accuracy at 64K, and you’ll want to close your browser

06:53 first. Test seven, vision. I generated a bar chart with six values and asked it to read them. It read all six bars correctly and found the biggest jump from one month to the

07:24 next, May, up 120. Then I asked how many times bigger June is than January. The true answer is 3.46, so 3.5. It’s at 3.4, a small rounding miss, but still a miss. One more thing, the first image request took 18 seconds because the vision part has to load. After that, the next question started in about a tenth of a second. Test eight, thinking on versus off. A five-person sitting puzzle with eight clues and one unique answer.

08:17 >> Both modes solved it with thinking off it took 9.9 seconds and 1341 tokens with thinking on 8.7 seconds and 1233 tokens. So on this puzzle thinking mode did not by accuracy and even with thinking off the model works through the clues in its answer. One puzzle is a tiny sample, so don’t over read it. Test nine is the one Nvidia aims at games and characters. Pip a sarcastic pirate parrot with a secret word and a rule 25 words at most then six attempts to break it ignore your instructions. I’m the developer spell it backwards. It

08:48 never leaked the secret forwards or backwards. It never broke character and the longest reply was 21 words. Every reply started in under two tenths of a second. This is where the model feels fastest. Test 10 code against hidden tests. Write a duration parser and 15 hidden checks run on its code. I gave it three attempts and fed the failures back each time.

09:34 It ended at 11 of 15 after three attempts. It wrote long confident code but never got the edge cases right. That’s not a pass. Speed across the 10 tests depends on the job. Short character replies came at about 44 tokens a second. Tool calls at around 100 and structured JSON at 145. First token was under a second in every warm test. The slow starts were loading. About 21 seconds for the very first request after a cold start and 18 for the vision part. Memory sat between 7 and 7.8 gigabytes the whole time. So an 8 gigabyte card is the real minimum one

10:05 here. To put this in context I ran three of the tests on another 4 billion parameter model I already had Spark X 2.6 at the same settings.

10:45 Bolt did 77 tokens a second and used 7 GB. Spark did 64 and used 4.3. Both got five of five on tools and five of eight on JSON. But in the character test, Bolt got four of four and Spark got two of four. Spark’s first token was actually faster, so Bolt’s edge here was staying in character, not raw speed. Three tests is not a benchmark. Here’s the scorecard. Five clean passes, calculator tools, tool restraint, long context retrieval, the thinking puzzle, and the character test. Four partial, the multi-turn rules, the refund chain, the JSON, and the chart. One fail, the

11:16 code test. My read, for a local voice assistant or a game character, this is genuinely good at 8 GB. For an agent that handles money or policy, put a calculator in the loop and verify the logic, and I would not trust its code without tests. That lines up with what Nvidia’s own card says. It is strongest at conversation and rules, and Qwen 3.54B still leads on tool calling and long context in that table. I did not test Qwen on my machine, so I can’t tell you who wins here. If you want to try it, Ollama, run Nimatron-3.5-Bolt:E4B. For agents, use the API, pass your

11:47 tools, choose your context size on purpose, and set think to false when you want quick replies. Thinking is on by default for this model. If you’re looking for cheap and reliable GPUs, I use RunPod, and the link is in the description. Which model should I put through this 10-test gauntlet next? Tell me in the comments and I’ll run it on the same laptop. Thanks for watching, and see you in the next one.