12GB Model, 8 Hours, One 3D Game: OrcaSAQ2 27B Tested

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: 12GB Model, 8 Hours, One 3D Game: OrcaSAQ2 27B Tested · Channel: Fahd Mirza · 11:23 · Watch on YouTube

Topics: AI agents, Software engineering with AI, Model inference and optimization, Local and open-source models, Open source and software ecosystems

Summary

The video tests OrcaSAQ2 27B, a 12.3 GB quantized version of Qwen 3.8 27B, served locally with vLLM and MTP speculative decoding. The host first runs a short multi-step coding task, which the model completes coherently in two turns. A much larger test asks it to build a 3D trampoline park game using Blender and Godot; the first version takes over five hours but has major rendering problems, and a second attempt only partially fixes them. The host concludes that the result is impractical and appears lower quality than the original model’s performance in a similar test.

Key points

OrcaSAQ2 27B compresses a 54 GB Qwen 3.8 27B model to 12.3 GB using a sensitivity-aware quantization method, according to the video.
The model is described as supporting 262K context, thinking mode, tool calling, and MTP speculative decoding.
The host serves the model locally with vLLM and reports that it uses under 12 GB of VRAM without KV cache, and 42–43 GB with KV cache.
A manually written script tests whether the model can maintain an agent loop of planning, acting, observing, deciding, and recovering.
The model completes the short coding task in two turns, reporting no issues and declaring success.
The larger Hermes agent test asks the model to create a playable 3D trampoline park game using Blender and Godot.
The first game attempt takes 5 hours, 3 minutes, and 54 seconds, but the player characters do not render correctly.
A second attempt produces a trampoline and some bouncing, but the host finds that spinning and several controls do not work.
The host judges the long game-building task impractical and says the original model produced a better result on its first attempt.

Chapters

Start	Chapter	Summary
00:01	Model overview and local setup	The host introduces OrcaSAQ2 27B, downloads it, and starts serving it with vLLM and MTP speculative decoding.
02:10	Quantization fidelity and agent-loop test	The video discusses the claimed fidelity of the compressed model and tests its multi-step reasoning with a short coding task.
06:34	Building a 3D trampoline park	The host gives Hermes agent a single prompt to create a Blender-and-Godot game, then inspects the first result after a long wait.
08:45	Inspecting the game and final verdict	The first game has serious rendering issues, and a second attempt only partially works, leading the host to call the process impractical.

Quotes

crushes it down to just 12.3 GB — 00:01

0.0.2 perplexity difference versus the full-size BF16 model. — 02:10

declared task success. — 05:59

It has taken 5 hours, 3 minutes, and 54 seconds. — 08:12

which is totally impractical. — 11:23

Concepts and entities

OrcaSAQ2 27B, Qwen, vLLM, Hermes Agent, Blender, Godot, quantização sensível à sensibilidade

Full transcript (original language)

00:01 The long tail of Qwen continues. Orca set takes Qwen 3.8 27 billion, a 54 GB model, and crushes it down to just 12.3 GB using a proprietary sensitivity aware quantization method. This new model supports 262K contacts, thinking mode, tool calling, and MTP speculative decoding. The target use case is long horizon agents, coding, and terminal tasks.

00:32 Today, we are going to serve it locally and put those claims to the test. This is Fahad Mirza, and I welcome you to the channel. And yes, I’m going to keep driving you crazy by telling you in every video that I have started this free weekly AI newsletter, so you need to subscribe there, please. Now, I have this Ubuntu system where I have this GPU card with 48 GB of VRAM, but you don’t need that much GPU. So, let me quickly download the model.

01:06 The model is now downloaded, as you can see, 12.3 gig. Now, let’s serve it with vllm. Look at the last line because this last flag enables MTP speculative decoding, which gives us up to 38% faster single stream decode speed. And I’m using vllm. If you don’t know how to get it installed, just go to my channel, search here vllm tutorial, and you should be able to find that video and get this vllm installed. Okay, so all it is telling me is that I

01:39 just need to install torch with my CUDA environment, so let me quickly do that. And I have installed it. Let me run that command again. Now, while it gets served, let me show you couple of interesting things, which actually tells us why this model is different from the original Quen. So look at this image here. This is quite interesting. They have squeezed this model down by 77% and claim it still behaves almost

02:10 identically to the original and we are going to test it out. Only 0.0.2 perplexity difference versus the full-size BF16 model. And 93.2% token level agreement means when you ask it something, it picks the same word as the original 93 times out of 100. So you still get 262K context, thinking mode, and tool calling, everything the full model has. This one is even more interesting.

02:43 This loop is exactly why fidelity matters for agent because if you go to the home page of this model card, it says that it is high fidelity 3-bit Quen. So this is what high fidelity is. An AI agent doesn’t just answer one question. It plans, takes an action, looks at the result, decides what to do next, recovers from mistake, and repeats the cycle over and over until the job is done.

03:13 Each step depends on the previous one. So if the model makes even a small wrong decision early on, that error carries forward and compounds across the entire task. That is why they obsess over keeping this compressed model as close to the original as possible. I’m going to use it with the harness. Doesn’t matter which one you use. People are asking me that I use Hermes agent a lot. I’m not fixated on that. I might actually switch from Hermes agent

03:44 because it has become quite heavy and lately quite clunky. And you know what? These days all these hardnesses are more similar than different. Anyway, doesn’t matter. But I think it will be a good use case to patch up both of these together to see how exactly they work. I might also do a test without any hardness just to show you this loop if it works out or not. Anyway, let’s go back to our terminal. And yes, by the way, if you’re looking to rent a GPU on very good price, you can find a link to vast

04:18 computing video’s description. Model is loaded through vllm and you can see that without KV cache it takes under 12 gig of VRAM. So, if you’re running this, make sure you have that much VRAM. And let me show you the VRAM consumption with KV cache for this model. 42 43 gig of VRAM with KV cache. Okay, so next what I’m going to do, I’m going to test this loop. I’ll just quickly show it to you in the browser which we did earlier.

04:51 So, this is the one I’m talking about and this is the script which I have written. So, this script is primarily sending a coding task to our model and forces it to work through the full agentic loop which is plan, act, observe, decide, recover, repeat until it declares task success. I have just created it manually. So, we are going to test whether this compressed three-bit model can maintain coherent multi-step reasoning across multiple turns without losing track of what it was doing. And

05:24 that is exactly the kind of long horizon thinking the Orca set two team built this model for. So, let me take you back and let me run this script here. I already have uh put it here. So, let it finish and then we will check the output. And this is the whole output and you can see the model is going turn by turn and completed the task in just two turns. In the turn one, it planned its approach, wrote a clean sleeve of an auto things implementation, observed its

05:59 own code, and started deciding if it was correct or not. So, you see, it is just doing that. And then came the turn two. It confirmed no issues, skipped a recovery because nothing was broken, and declared task success. What you’re looking here is 12 uh GB compressed model maintaining full context and coherent reasoning across multiple steps. This is exactly, as I said earlier, what long horizon agentic behavior looks like. Let’s now pack it up with Hermes

06:34 agent and do some uh harder tests. And now, let’s do first test. Let me paste it, so you can see that it is quite a comprehensive goal. We are going to give the model a single prompt and ask it to build a complete 3D trampoline park game entirely on its own. It should be using Blender to create all the 3D assets through code, export them, and then build a playable game in Godot with bounce physics, flips, a combo

07:05 score system, and multiple zones. This is a very comprehensive long horizon task. I will run it. While it runs, let me show you that when it who are you, it gave me the response in short, but look at the reasoning. It seems it doesn’t really resolve the overthinking um of Gwen, which is fine, because that is not what this model is claiming. It is actually It actually shows that it is more closer to the original than the other uh fine-tuned or other variants which we

07:37 have already tried. Anyway, and look at the number of commits these Hermes uh you know people are doing. I don’t know why, but this is really crazy and needless. Harness should be lightweight. No need to push thousand and thousands of commits on every day. Anyway, I digress. So, I will let this one finish and then we will check out the result. Okay, eventually this three-minute compressed model has come back and you can see from the output it says it has built a complete 3D trampoline park game

08:12 entirely on its own. All the blender assets, Godot game engine, bounce physics, flips, scoring, six zones, all from a single prompt. But, let me show you something before I proceed and show you the actual thing. There you go. It has taken 5 hours, 3 minutes, and 54 seconds. Um even my shave has grown back. Back. It is so so long. Wait, you won’t even believe. Anyway, so let’s see if it was uh worth the wait or not. Let me launch

08:45 this. It has asked me to go to this path. Um okay, so I Okay, so I’m not in the right directory. Let me go to the right one. So, this is the directory with all the assets. Let me now launch this. And there you go. This is what it shows on screen. No rendering of players, no rendering of actual um characters on screen. I’m also pressing some of the keys and you see it says that something is happening like I

09:16 quickly pressed a couple of them and it bounced. Bounces and then spins, but the actual stuff doesn’t really render at all. You see there’s a score two. And it is totally off at the moment. So, it took more than 5 hours and couldn’t do the job in first shot. So, what I’m going to do, I’m just going to go back to Hermes plus this model and tell what exactly is not working and see if it is able to fix it. Last and final chance for the model. So, this is the

09:48 time’s up. So, the game is working, but the rendering is not there, which was actual thing. So, let me go back to the terminal. This is the result of second attempt. Let me start it. Okay, now this time it works. Okay, so this is the trampoline. Something is happening. Let me make it bit bigger if I can. Where is my mouse? It is stuck. Anyway, I’ll press left arrow 360 spin.

10:18 At least it is bouncing. We have the trampoline. And you see I just uh quickly clicked upper arrow and it bounces couple of them. Is it spinning? I don’t see it spinning. Was it too fast for me? So, it is there, but of course I mean it’s not there yet. So, I’m just doing it again, again, again, again. It’s not working. Let me Can I make it bigger? Okay, it’s not letting me make the screen bigger.

10:50 Now, I’m just going to press E here. Nothing happens. W capital W, nothing. So, some of the controls are not even working. Timer is up. There you go. So, my score is this. So, I’m giving you as is. It happened. So, look I think there is a lot of uh loss of quality because the original coin took same time, but it was able to deliver much better result in first go. And it took me around 8 hours to make this video just for this single test,

11:23 which is totally impractical. Anyway, this is uh the high fidelity for you. Let me know your thoughts in the comments. And I will go uh just you know take some rest I guess because my back is broken after looking at it for 8 hours. Anyway, if you want to help out the channel, please become a member. Thank you for all the support.