Spark-X2.6-4B's CRAZY Thinking Dial: 180 Tests on an 8GB GPU

Author: Prompt Engineer 48 · Watch on YouTube

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: Spark-X2.6-4B’s CRAZY Thinking Dial: 180 Tests on an 8GB GPU · Channel: Prompt Engineer 48 · 10:17 · Watch on YouTube

Topics: AI evaluation and benchmarks, Model inference and optimization, AI infrastructure and hardware, AI models, Local and open-source models

Summary

The video tests Spark X 2.6 4B’s three thinking modes—none, low, and high—on an 8 GB laptop GPU across 180 runs of 20 graded challenges. Low and high improved logic-puzzle accuracy over none, but high used substantially more tokens and did not outperform low overall. None was fastest and performed well on debugging and tool tasks, while thinking sometimes caused long runs or failures. The reviewer recommends low as the default, none for simple tasks, and high only when low fails, with a token cap.

Key points

The tested model is about 4.1 billion parameters; the reviewer used the official Q4_K_M GGUF file, about 3.5 GB, on an RTX 4060 laptop GPU with 8 GB of VRAM.
Across 60 runs per mode, accuracy was 48/60 for none, 54/60 for low, and 53/60 for high.
Low and high both solved all 15 logic-puzzle runs, compared with 8/15 for none.
Debugging results favored none: 15/15 correct, versus 13/15 for low and 12/15 for high.
Thinking did not improve exact 50-word story writing; all modes scored 0/9.
Total generated tokens were about 12,000 for none, 48,000 for low, and 69,000 for high.
High used about 45% more tokens than low without improving the overall result in this sample.
The reviewer recommends low as the default, none for simple tasks, and high only when low fails, using a token cap.

Chapters

Start	Chapter	Summary
00:00	The thinking dial and test setup	The reviewer introduces the none, low, and high modes, describes the model and GPU setup, and explains the 20 graded challenges.
02:50	How thinking works and overall results	The video explains the prompt-template difference between modes, then compares accuracy, token use, and run times across 180 runs.
05:31	Category results and examples of failures	The reviewer breaks results down by task type and shows cases where thinking helped, made no difference, or led to failures.
08:41	Using the dial and final recommendations	The video covers the Ollama and API settings, offers practical mode recommendations, and notes the limits of the test.

Quotes

With thinking off, it answered 50 in a third of a second. — 00:00

So none isn’t a lazy version of thinking. It skips the thinking step entirely. — 02:50

High spent about 45% more tokens than low and bought nothing extra in this sample. — 04:27

So, a 4-billion model can’t count words in its head, and thinking doesn’t fix that. — 07:35

Use low as your default. — 09:12

Concepts and entities

Spark-X2.6-4B, Ollama, llama.cpp, Hugging Face, X Open, GGUF, quantização de 4 bits, RTX 4060, token cap

Full transcript (original language)

00:00 I asked a 4-billion parameter model, a classic trick question, a bat and a ball cost a dollar 10 together, and the bat cost a dollar more. How many cents is the ball? With thinking off, it answered 50 in a third of a second. Same model, same question, I flip one switch to low, and it writes out the algebra and says five, which is correct. And that is the problem with small local models on a laptop. Thinking off is fast and sometimes confidently wrong. Thinking on is smarter, but every thinking token is time you spend staring at an 8-GB graphics card. Spark X 2.64B says it fixes that with a three-position

00:35 dial, none, low, and high. So, here’s the only question I care about. Does turning that dial up actually earn its tokens? I ran 20 graded challenges in all three modes three times each. That’s 180 runs on this laptop, and I’ll show you every failure. >> [music] >> First, what is it? This is the model card on Hugging Face, Spark X 2.64B, published under the X Open account, Apache 2.0 license, about 4.1 billion parameters. The authors say it keeps exactly the same architecture as Spark X

01:07 2.5, and that the gains come from better post-training. Their pitch is small models for on-device agents, coding tool use, and long context. And this is the line that made me want to test it. Controllable thinking budget, three levels, none, low, high. They say it lets you balance reasoning quality against latency and cost. They also published these benchmark charts comparing it with QN 3.59B and Gemma models. Those numbers are developer reported. I did not reproduce them, so treat them as the authors claim, not my result. One fact check on the brief I started from, it said the original release date isn’t established, and I can confirm that. The repository was

01:39 created on October 19th and last updated on October 21st. That’s repository history, not an official release date. Now, can my machine run it? This is an RTX 4060 laptop GPU with 8 GB of video memory. That is the whole budget for the model and for everything it thinks about. The model card says to run it with Ollama, run Spark LLM/Spark X 2.64B. I pulled that exact tag. Look at the tag page first though. That default tag is 8.2 GB. It’s the full 16-bit model. It runs, but look at what Ollama reports. 9.5 GB at its default context. Split 49% on the CPU and 51% on the GPU. It

02:15 generated about eight tokens a second. The better option is the official GGUF repo. The Q4 KM file is about 3 and 1/2 GB. Fully on the GPU, it generates around 60 tokens a second. That’s roughly eight times faster though I’m comparing two run times. It’s what I used for everything else. You do need a recent run time. The card says llama.cpp build 10828 or later and Ollama 0.34 or later. I’m on newer builds of both. Loaded with a 16,000 token context, the model takes about 3 and 1/2 GB of video memory. 3,543 MB to

02:50 be exact. It fits comfortably. Here’s how the dial actually works and it’s simpler than I expected. This is the chat template that ships with the model. If thinking is switched off, the template ends the prompt with a close thing tag. The model starts its answer as if it has already finished thinking. So none isn’t a lazy version of thinking. It skips the thinking step entirely. If thinking is on, the only difference between low and high is one sentence in the system prompt. Now the system is invoked with thinking effort equals low or high and high is the default.

03:21 That’s worth knowing because it means the budget is a suggestion the model follows, not a hard limit. The only hard limit in my runs was my own token cap, which I’ll come back to. Here’s the test. 20 challenges, five each in four categories. Debugging broken Python, calling tools, logic puzzles, and following exact formatting instructions. Nothing is graded by me reading answers. Debugging answers are code that I execute against tests. Tool calls are parsed and the arguments are checked. Logic questions need the exact final number. Formatting rules are checked in code. I used the settings from the model card.

03:51 Temperature one, top P 0.95. Each challenge ran in all three modes and I repeated the whole set three times because at temperature one, a single run is a the flip. Generation was capped at 6,144 tokens. That’s 60 runs per mode, 180 in total, one request at a time, so the timings are clean. I did have to throw away an earlier attempt where two of my own runners overlapped on the same server and redo it from scratch. Here are the totals first with thinking none 48 of 60 correct, low 54, high 53. That’s 80%, 90%, and 88%. Low and high

04:27 are one test apart with three runs each at temperature one. I would not call that a difference. The cost is where it gets lopsided. Across all 60 runs, none generated about 12,000 tokens, low generated 48,000, almost four times as many, high generated 69,000, nearly six times. In time, none took 5 minutes in total, low took 23, high took 36 minutes. The typical question is much calmer. Median time is about 1 second with none and 4 to 5 seconds with low or high. The averages are dragged up by a handful of very long thinking runs. So, high spent about 45% more tokens

05:00 than low and bought nothing extra in this sample. Here’s a fairer way to see the cost. Divide the tokens by the number of correct answers. None spent about 250 tokens per correct answer, low spent about 885, high spent about 1,300. That is what a right answer actually costs you in each mode. One more thing about that dial. I have 60 matched pairs, the same challenge in the same pass, once on low and once on high. High thought longer 39 times, low thought longer 17 times, and four were equal. So, high is a lean toward more thinking, not a guarantee.

05:31 And here’s the waiting in plain numbers. Runs that took more than 30 seconds, three with none, 11 with low, and 17 with high. Those slow runs are what the averages are made of. Now, the full grid. Every challenge, every mode, three runs each. 12 of the 20 challenges were three out of three in every mode. Thinking made no difference to them at all. Everything interesting is in the other eight rows, so let’s go through them. Now, where does thinking actually help? Let me split it by category. Each cell is out of 15 runs. Logic is the clear winner. None got eight of 15, low and

06:03 high both got all 15. For reasoning puzzles, thinking is clearly worth it. Instruction following is a flat line, 12 out of 15 in all three modes. Thinking didn’t move it, and I’ll show you why in a moment. Tool calling, 13 for none, 14 for low and high. That’s a single test making the difference, and I’ll show you that one, too. The rest of the tool tests are worth a note. Asking for the time and the weather in Tokyo needs two tool calls at once, and every mode got that three times out of three. A simple factual question should not trigger a tool at all, and every mode had that back three out of three. A tool run average 24

06:35 tokens with none, 60 with low, and 97 with high. For tool calling, none is enough. And debugging surprised me. None got 15 out of 15, low got 13, high got 12. Thinking did not help here, and in this sample, it even hurt a little. The thinking didn’t write worse code, exactly. High’s three misses, two ran out of tokens while still thinking, and one wrote code that never finishes. Low’s two misses were plain wrong fixes. Longer thinking just gives a small model more room to wander. And debugging is where the token bill is biggest. An average debugging run took

07:05 72 tokens with none, 1,100 with low, and 2,500 with high. So, the mode that got 15 out of 15 was also the cheapest by wide margin. Let’s look at the failures word for word. Bat and ball, thinking off, the model’s whole answer is 50. Another run said 10. Across three runs, none never got it. Low and high got it every time because they write the algebra out first. The farmer puzzle, where all but nine sheep run away, is the same story. With thinking off, it said eight or 10, never nine. With thinking on, it solved it

07:35 every time. Now, the opposite, a broken binary search. In high mode, the model thought for about 17,000 characters, hit my token cap, and returned an empty answer. Low solved this one three times out of three. High solved it once. Then there’s this one, write a story of exactly 50 words. Zero out of nine in every mode. With thinking off, it didn’t even stop. It rambled for 1,400 to writing things like, “Wait, need exactly 50. Let me count properly.” Low and high ran out of tokens counting. So, a 4-billion model can’t count words in its

08:09 head, and thinking doesn’t fix that. One tool test needs an honest note. I asked for Python files starting with test underscore. It used a pattern test star.py or star test star.py. That’s close, but looser than I asked, and my grader marked it wrong. You might call it a pass. I kept my strict grading, and I’m telling you speed is the same in all three modes, about 60 tokens a second. That means time is just tokens. If high thinks twice as long, you wait twice as long. Nothing else changes. Memory is the same, too, about 3 and 1/2

08:41 GB, whichever mode at that 16,000 token context. You don’t need a bigger card for high. You need more patience. If you use Ollama, the same dial exists. I tested it with the 4-bit model. With think set to false, it answered in six tokens and said 10 wrong. With think low, 92 tokens and the right answer. With think high, 120 tokens, also right. In the API, that’s the think field. False, low, or high. Setting it to true behaves like high, which matches the template where high is the default.

09:12 One practical thing from that earlier screen, Ollama loaded the full model with a context of 65,000 tokens, and that context takes memory, too. Shrinking it is the first thing I’d try if you want the big version to fit better. I haven’t tested how much it helps, so that’s a suggestion, not a result. So, should you turn thinking up? Here’s what these 180 runs say. Use low as your default. It fixed every logic trap, and it used about 70% of the tokens that high did. Use none for things that should be instant, tool calls, short code fixes, formatting. It was by far the fastest, and in my test, it it was fine on

09:46 debugging, but don’t trust it on a trick question or a word problem. Use high only when low fails, and put a token cap on it because high is where I saw the runaways. And here’s what this can’t tell you. One, 4-bit quantization. 20 tests I wrote myself, three runs each at temperature one, one strict grader, and the vendor’s benchmark charts are still the vendor’s claim. I didn’t rerun any of them. If you’re looking for cheap and reliable GPUs, I use RunPod and the link is in the description. If you want me to test the other spark

10:17 quantizations or push this against a bigger model, tell me in the comments. See you in the next one.