Ternary Bonsai 2 9B in GGUF: A 2.4 GB Model Locally

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: Ternary Bonsai 2 9B in GGUF: A 2.4 GB Model Locally · Channel: Fahd Mirza · 09:24 · Watch on YouTube

Topics: AI evaluation and benchmarks, Model inference and optimization, AI models, Local and open-source models, Multimodal models

Summary

The video reviews Prism ML’s Ternary Bonsai 2 9B GGUF model, which is presented as a roughly 2.4 GB compressed version of Qwen 3.5 9B that supports image and text input and retains a 262K context window. The host explains how ternary weights reduce the model’s size, then tests coding, Nadsat, multilingual output, scientific reasoning, and image-based creative writing. The model performs well in some image understanding and coding tasks, but shows weaknesses in language accuracy, numerical details, and plot logic. The host concludes that it is a promising effort, while warning that its exact answers and creative output should not be fully trusted.

Key points

The opening demonstration shows the model generating code for an animated donair kebab skewer after being given a photo and prompt.
The 9B GGUF model is described as about 2.4 GB, compared with about 17.9 GB for the original Qwen model.
Ternary weights use three values: -1, 0, and +1; a small-scale number is stored for each group of 128 weights.
The model supports images and text and retains a 262K context window; the host notes that lowering the context length reduces VRAM use.
In the Nadsat test, the model used some real words but often used them incorrectly.
In the 28-language test, output quality generally declined for less common languages, with some collapsing into repeated nonsense.
The scientific image test reached a sensible overall conclusion, but included misread numbers, an unsupported stall angle, and arithmetic errors.
In the situational image test, image description was strong, but the sketch had shaky Urdu, repetition, and a confusing plot.
The host considers this model an improvement over previous models in the family, while noting a substantial performance loss compared with a larger model.

Chapters

Start	Chapter	Summary
00:01	Introduction and model size	The host introduces a code-generation example and outlines the model’s basis, image capabilities, context window, and compressed size.
02:07	Ternary weights and the Nadsat test	The host explains how ternary weights shrink the model, shows its runtime setup, and evaluates its use of A Clockwork Orange slang.
04:44	Multilingual and scientific reasoning tests	The model handles major languages more successfully than rarer ones, then reviews a wind-tunnel infographic with a sound overall conclusion but shaky details.
07:19	Situational test and final assessment	The host finds the model’s image description strong but its creative story weak, and closes with a comparison to a larger model.

Quotes

Ternary means we squeeze every one of those just into three values, -1, 0, or +1. — 02:07

I don’t think so it is really speaking net set — 04:11

Quality drops as the language gets rarer. — 05:46

you should never trust its exact numbers at all. — 07:19

it’s a good image reader and a decent Italian speaker, but weak, um, at plot logic. — 08:52

Concepts and entities

Ternary Bonsai 2 9B, Qwen3.5 9B, GGUF, quantização ternária, llama.cpp, Fahd Mirza, Nadsat, Prism ML

Full transcript (original language)

00:01 You are looking at a donair kebab skewer spinning in front of a gas flame running live in my browser. Now the interesting part, a model with only 2.4 GB of weights wrote this code. It first looked at a real photo of a donair, then built the whole animation from that. Let me show you where it did that. So you see, I provided it this prompt and this photo, and then it reasoned. After the reasoning, it generated this code which

00:34 I just showed you. This is Fahad Mirza, and I welcome you to the channel. I have been covering these bonsai models for the long time, I would say from the day one. And they are hit and a miss. Sometimes they are great, and sometimes they just totally fail at, you know, don’t work at all on any prompt. So I’m very curious to see how exactly this new bonsai 2 9 billion performs. You can see that the coding was really a

01:04 good one. Let’s test it more. Before that, let me give you a very quick intro as what exactly these ternary models are and what this prism ML is doing with these ternary bonsai models. So this one is a 9 billion in GGUF format as you can see. They also have other smaller variants. It is built on Qwen 3.5 9 billion. So it understands images as well as text, and it keeps the huge 262K context window intact.

01:36 The original model is about 17.9 gig, the Qwen I mean. This one is about 2.4 gig, that’s it. That is roughly seven times smaller, and the weights are Apache 2, so you can use them freely. They also have shared some of the benchmarks which I I be showing you shortly, but before that, I just want to recap how exactly they have shrunk this model this much. On the left is 13.59

02:07 billion, the original. Inside every layer, each weight is a precise 16-bit number like .82 or - .07. Ternary means we squeeze every one of those just into three values, -1, 0, or +1. That is it. Three choices only. A small-scale number is stored for each group of 128 weights to keep the sizes right. Fewer bits per weight means a much smaller file, and that is how 17.9 gig

02:38 turns into about 2.4 GB. So, look. This is where it is running with llama.cpp, as I mentioned earlier, and this is a VRAM consumption at the moment with full 262K context window. If you don’t want that much, just reduce that context length here by just changing this C parameter, this VRAM consumption will come down. And yes, I’m using a bigger GPU because I’m just reusing this VM.

03:09 If you’re looking to rent a GPU on very good price, you can find the link to Mass Compute in video’s description. Please also subscribe to this free weekly AI newsletter at fahadmirza.substack.com. And if you want to help out the channel, just go to the home page or in the first pinned comment and click on this link to become a member. Okay, so that’s out of the way. Let’s test this thing bit more. I’m going to ask it another interesting question. Now, this prompt asked the model to

03:40 retell a real historical event in Nadsat, which is a made-up slang from A Clockwork Orange to test whether a heavily compressed model still knows real Nadsat or just turns into poetic sounding text. By the way, the reasoning is pretty interesting of this model. Despite of its size, it actually thinks quite nicely. Anyway, what depends on the success is the result. So, let’s wait. Okay, so it has produced the result. And

04:11 again, I would say it’s a 50/50. It used real net set words, but mostly I think with wrong meanings. Only drug, malchik, and moloko are right, but there are various which are wrong. So, I would say that is why I’m saying maybe 60/40 or 50/50 result. But all 11 words came from the prompt two, by the way. So, it is sprinkling the words in. I don’t think so it is really speaking net set, but if you know if it is right or wrong, let me know in the comment. Also, this idea actually came from a

04:44 member of the channel. So, I would actually wait for his feedback to see if the result the result is good or not. Let’s do another test. Next test is all about multilinguality. So, look at the prompt. I gave the model one restaurant scene and asked it to reply in 28 languages across the continents. Big languages like Spanish and Portuguese in the result have come out quite strong and also Mandarin, I would say. But I think there are small slips in

05:15 Korean and also Indonesian. And then mid-resource languages like Hindi, Urdu, Greek, and also a few of the African ones came out garbled with stray Russian and Chinese words leaking in. And then there are some rare languages, too. Let me just first scroll down. They see they are okay, but again, European ones are more mostly okay, I would say. And these are some of the North American. But you see, all of these rare ones, they have collapsed into repeated nonsense.

05:46 Uh, think this was unfair to model two. It’s a two-bit model. But anyway, we just wanted to stretch it. So, overall, I would say the pattern is very clear. Quality drops as the language gets rarer. So, this is multilinguality for you. Okay, now let’s do a scientific reasoning one. So, for this test, I showed the model this busy wind tunnel infographic and asked it to review it like a senior aerodynamicist. Read the numbers, check them against each other, and then also check out the

06:18 physics and some maths and few other things as you can see from this prompt. Now, you know what? Quickly have a look at this reasoning. I actually enjoyed going through this reasoning. This is a very very well-crafted one for this condensed model. It didn’t really miss a beat. But then, if you look at the response, what it has done is that it has got it right, which is really impressive. It spotted conflicting Reynolds number,

06:49 called out garbled text and nonsense formulas, cuz that was an AI-generated one, and ruled the report non-credible, which is a fair call for this image full of gibberish. But the details were shaky. Especially here, what you [clears throat] are looking at your screen. It misread small numbers. Um, and it invented even a stall angle of less than 90 that I cannot find in the image at all. Its own physics check also had an arithmetic slip with 100 m/ Sorry, mi/s

07:19 and some other numbers which were really not correct. So, I think a 2.4 GB model can reason about a complex chart and reach a sensible conclusion, but you should never trust its exact numbers at all. So, I think hallucination is still there. Um, but it has improved as compared to the previous models in this family. Okay, now let’s do a situational test. So, you see, I gave the model this

07:50 AI-generated photo of a rich bachelor with a big gravy stain on his shirt trapped between two rival would-be mothers-in-law, an Italian nonna holding a lasagna and a baby photo and a Pakistani aunty holding a roast chicken and a family album. Each wanting him to marry her daughter. So, I have asked the model in this prompt, which you can see on your screen, to describe this image, then write a funny sketch with real cultural touches, Italian Urdu phrases, and a

08:21 twist ending. Now, if you look at the response, um, you know, reasoning is also quite interesting. Anyway, if you look at the response, the vision part was excellent. It described every person, dish, and the stain correctly. And the Italian phrases were fine, too, with one genuinely funny line, “That don’t look at the lasagna like it owes you money.” But, then the story really fell apart. The Urdu was shaky, um, really, I think it was not really good enough, and quite repetition there. The

08:52 daughters and the marriage were never mentioned at all. The man suddenly wore a suit instead of his blue shirt, and the baby photo twist made no sense at all. So, look, I would say it’s a good image reader and a decent Italian speaker, but weak, um, at plot logic. So, creativity is not its forte at all. So, look, this is in front of you, the ternary bonsai. There have been quite a heavy loss in performance, as you can see, especially if you have seen my 23.59 billion video. Check it out and then compare it with

09:24 this video. You would know what I’m talking about, but a good effort from Prisma ML. Let me know your thoughts. Thank you for all the support.