Nanbeige4.3-33B Is CRAZY Good (On Paper): I Tested It on a Rented GPU

Author: Prompt Engineer 48 · Watch on YouTube

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: Nanbeige4.3-33B Is CRAZY Good (On Paper): I Tested It on a Rented GPU · Channel: Prompt Engineer 48 · 09:57 · Watch on YouTube

Topics: AI agents, AI evaluation and benchmarks, Model inference and optimization, Local and open-source models

Summary

The video tests Nanbeige 4.3 33B, an Apache 2.0 model whose benchmark results look strong but have not been independently reproduced. The reviewer rents an RTX Pro 6000 with 96 GB of memory, gets the model running, and finds that its claimed 512K context window is not practical on that single card in BFloat16. In hands-on tests, it handles a math problem, catches a false premise, makes appropriate tool calls, fixes a small Python project, writes a working Snake game, and retrieves passcodes at up to about 95K tokens. The model is slow and prone to overthinking for one user, but parallel requests make its throughput and estimated cost per token more appealing.

Key points

The model card reports strong results on several agentic benchmarks, including Terminal Bench 2.1, but the reviewer did not reproduce them and notes that the lab selected the evaluation setups.
Nanbeige 4.3 33B has 33 billion parameters, Apache 2.0 licensing, BFloat16 weights of about 62 GB, and a claimed context window of 512,000 tokens.
The reviewer rented an RTX Pro 6000 with 96 GB of memory; the pod creation price was $2.49 per hour, above the catalog quote of $1.69.
Serving at 131,000 tokens failed because the KV cache needed more memory than remained after loading the weights; the reviewer ran the model at around 106,000 tokens instead.
At one request at a time, generation was about 21.8 tokens per second, and the model used its full thinking-token limit on a simple hash-map question without producing an answer.
It answered a modular arithmetic question correctly, rejected a nonexistent Python function, and made weather tool calls in parallel while waiting to use the calculator.
In a small shell-based coding task, it fixed three bugs; the reviewer then ran the tests and got three passes.
It produced a working Snake game in one HTML file, though the reviewer did not test every requested rule.
It found hidden passcodes in tests at 22,000, 71,000, and 94,867 tokens, but the reviewer cautions that this is a simple long-context test.
With thinking off, 32 concurrent requests reached 534 tokens per second total; the reviewer estimates roughly $1.30 per million output tokens for short-story generation at the pod price.

Chapters

Start	Chapter	Summary
00:00	Model claims and benchmark results	The reviewer introduces Nanbeige 4.3 33B, explains its teacher-model role, and assesses the model card’s benchmark results and caveats.
02:35	Renting a GPU and getting the model running	The reviewer compares GPU memory and pricing, loads the weights, and explains why a 131K-token context failed while a shorter one worked.
04:40	Reasoning, tools, coding, and a game	The model is tested on simple questions, tool use, a small Python bug-fixing task, and a one-file Snake game.
07:22	Long context, batching, and verdict	The reviewer reports passcode retrieval results, measures concurrent request throughput, and explains who may find the model useful.
08:24	Limitations and recommended settings	The reviewer lists untested areas, shares suggested serving settings, and concludes with the cost and an invitation to suggest future tests.

Quotes

The license is Apache 2.0. — 00:00

So, the 512K context not on one card, not in BFloat16. — 03:38

It called the weather tool for Tokyo and Paris in parallel and held the calculator back because the calculator needs the results first. — 05:43

534 tokens a second is about 1.9 million tokens an hour. — 07:52

If you want a self-hosted engine for batches, agents, or a pipeline, it’s very interesting. — 08:24

Concepts and entities

Nanbeige 4.3-33B, Nanbeige, Boss Zhipin, Terminal-Bench 2.1, Qwen, vLLM, RunPod, RTX Pro 6000, BFloat16, KV cache

Full transcript (original language)

00:00 On Terminal Bench 2.1, a benchmark where an AI agent has to actually work inside a command line, this new open model scores 80.9. The Qwen model it’s compared against scores 73, and the license is Apache 2.0. Now, the catch. This thing is 33 billion parameters. Just loading it takes 62 GB. My laptop’s graphics card has eight. So, right now the people who get to use agents this good are mostly the people renting them by the token. This is Nanbei Ge 4.3 33B. So, I rented a GPU, loaded it, and tried to break it. I’ll show you what worked, what didn’t, and one thing the model

00:31 card does not tell you. >> [music] >> Here’s the model card. Nanbei Ge is a lab inside Boss Zhipin, a Chinese recruiting platform, and this is their flagship built as a teacher. The idea is you train one big strong model, then use it to distill smaller ones. They’ve already said a 3B student is coming. Why should you care about a teacher model, even if you never run it? Because many of the small models you can run at home are trained from big ones. A 3B student of this teacher is small enough to fit on a normal graphics card. So, even if

01:01 33B is out of reach, what it teaches might not be. The files tell you more than the marketing. It’s a plain Llama architecture, 52 layers, eight key value heads, Bfloat16 weights, and a claimed context window of 512,000 tokens. License Apache 2.0. So, no weird usage terms to read. Now, the benchmarks, and I want to be clear every number on this table is the lab’s own. I did not run these. They say all models were evaluated at their highest thinking effort, and this one ran with preserved thinking switched on.

01:32 On the agentic coding rows, it leads Terminal Bench 2.1 80.9 against Qwen’s 73. Deep SWE 65.5 against 42.5. That’s a big gap if it holds up outside their setup. The second block is what they call agentic co-work office-style tasks. Automation Bench 54.2 against 38.3, Job Bench 44.8 against 34.5, and YC Bench, where a model runs a simulated startup, it ends with about 1.02 million dollars against 641,000 for QN. But, read the whole table. On Stribench Pro, it loses

02:03 60.3 to 61.7. On GDP Val, it loses. And on GPQA Diamond, Gemma beats it 91.3 to 89.5. It’s strong, not unbeatable, and I like that the card shows its own losses. One more thing before you trust any of this, read the footnotes. The card says every model gets the same harness on each benchmark, which is fair. But, terminal bench uses one harness, deep sway in another, and NL2 repo runs through cloud code. The lab picked those setups, and I haven’t seen an independent reproduction yet. That’s why I wanted to test it myself. So, why not just run it here? Let me show you, this is my laptop, an RTX 4060 with 8 GB of

02:35 video memory. The weights alone in 16-bit precision are about 62 GB. It isn’t close, so it’s a rented GPU. The rule I use is weights plus headroom for the KV cache, which is the memory the model uses to remember your conversation. The model card doesn’t give a minimum, only a serve example with four GPUs. So, I wanted one big card to keep it simple, 96 GB. Here’s the live capacity data from Runpod’s API, not typed by me. The A100 has 80 GB, which leaves roughly 18 for the cache, tight. The RTX Pro 6000 has 96, and it was the one showing high stock. That’s the one I picked. Now, price

03:06 honesty, the catalog said $1.69 an hour. When I created the pod, the response said $2.49. That’s almost 50% more than the quote. I kept it because the stock was there, but check the create response, not the list price. Settings are simple, a PyTorch image, 220 GB of disk so the weights fit, port 22 for SSH, and port 8000 for the API. I connected, ran nvidia-smi, and there it is, 97,887 MB. Setup was fast. I installed vLLM version 0.32, and downloaded 62 GB of

03:38 weights in under 2 minutes. Then, I tried to serve it at 131,000 tokens of context, and it crashed. The error is the interesting part. At that length, the cache needs 26 GB, and after loading the weights, I only had 21.4. vLLM estimated the most this card can hold is about 108,000 tokens. So, the 512K context not on one card, not in BFloat16. And I could have predicted that crash with a calculator. For every token, the cache stores a key and a value for 52 layers, eight heads, 128 numbers each,

04:09 two bytes per number. That’s about 208 kilobytes per token. Times 131,000 tokens is 26 GB bytes. At 512K, it’s over 100. So, I relaunched at 106,000. It came up and the cache holds 108,000 tokens, just enough. Everything after this runs on that setup. Remember the number 95,000. We’re going to hit it. First test speed. I asked it for a 300-word explanation of how a hash map handles collisions. Simple. It generated 21.8 tokens per second, which is normal for a dense 33B model since every token reads all 62 GB. But look at what

04:40 happened. It spent all 4,000 tokens, my limit, just thinking. It never wrote the answer. 3 minutes of thinking for a hash map question. This is a thinking model by default. And on easy questions, it overthinks. You can turn it off, and I did later. Second test, a math question. How many ordered pairs of numbers from 1 to 100 have a squared plus b squared divisible by 7? It answered 196 in just under 27 seconds. I checked that myself. Squares mod 7 can only be 0, 1, 2, or 4. So, the sum is 0 only when both are multiples of 7. 14 multiples. 14 * 14 196. Correct?

05:12 And it took only 592 tokens. So, thinking isn’t always long. Third, a trap. I asked it how to use the quantum sort function from Python’s create module to sort in That function doesn’t exist, and a constant time sort is impossible anyway. You’d have to read the list. It caught both. It said there’s no quantum sort in iterate, showed a one-line check that proves it, and gave me the normal sorted function. One more thing it did right. It said that constant time only works if the list already sorted. That’s the kind of honesty I want. Next, tool calling. This matters because agents live or die on it. I gave it two tools, a weather

05:43 lookup and a calculator, and asked for Tokyo and Paris weather, then the difference times three. It didn’t fire everything at once. It called the weather tool for Tokyo and Paris in parallel and held the calculator back because the calculator needs the results first. That’s correct dependency reasoning. I’ll be straight though, I only tested this first turn, not a full loop. So, the real agent task, I made a tiny Python file with three bugs and a test file and gave it a shell. The bugs were a discount divided by 10 instead of a 100, a total that ignored quantity, and a top item picked by unit price. It ran four steps, read the files, rewrote them, noticed that Pytest wasn’t

06:14 installed and ran the test another way, then summed up the three fixes. I ran Pytest myself afterwards, three passed. Now, honest limits, this is a toy. Three bugs in a tiny file is nowhere near Terminal Bench. It shows the loop works. It reads, acts, checks. It doesn’t prove the 80.9. Then the stress test, I asked for a complete snake game in one HTML file with scoring speed increasing every five foods, a high score saved in the browser, and restart on space. This took 22 minutes, 27,969 tokens, and 17,000 of them were thinking. But when it finished, I opened the file and the game just worked, no

06:44 console errors. Now watch, this is the game it wrote and the key presses are scripted by me, so I could film it. The code has the local storage line for the high score and the speed step every five foods. I didn’t test every rule, but it runs. It looks polished and it’s one shot.

07:22 Next, long context. I hid a passcode inside a pile of filler text and asked for it back at 22,000 tokens. It found it at 71,000, found it at 94,867 tokens, found it. That’s three out of three and it answered in 44 seconds at the longest. But a needle test is the easiest long context test there is and I could only go to 95,000 not 512. So this tells you it holds up to there, nothing more. Now the part that changed my mind, everything so far was one user at a time, 22 tokens a second. So I sent

07:52 eight requests at once with thinking off, 157 tokens per second total. Then 32 requests at once, 534 tokens per second, same card, same model, 24 times the output because the weights get read once and shared across all of them. Do the math with me. 534 tokens a second is about 1.9 million tokens an hour. At $2.49, that’s roughly $1.30 per million output tokens. That’s my back of the envelope for short stories, not agent work, but it shows who this model is for. So here’s my verdict. If you want one private chat window on one GPU, this is slow and it overthinks. 22 tokens a

08:24 second is not fun. If you want a self-hosted engine for batches, agents, or a pipeline, it’s very interesting. Quick decision guide, if you’ve got a normal gaming card, 24 GB or less, don’t try the full model, wait for the 3B student, or test the community quantized versions yourself. If you have one big server card, it runs, but expect around 20 tokens a second per user. And if you have lots of parallel work like a batch of tickets or agents running side by side, that’s where the price per token gets interesting. What I didn’t test, I didn’t reproduce any of their benchmarks, I didn’t try the 36 community quantized versions

08:56 listed on the page, and I didn’t go past 95K tokens. If you try the quantized ones on a smaller card, tell me what you find. What I did prove, it runs on 196 gig card, it follows tool calls, it fixes real bugs, it wrote a working game, and it caught a false premise. For an Apache license model, that’s a real result. If you want to run it yourself, here’s what the card recommends. Temperature 1.0 for agent and tool work, 0.6 for reasoning and chat, turn on preserve thinking for multi-turn agent loops, and turn it off for normal chat. Ask for the XML tool call format and

09:26 serve it with the Qwen 3 reasoning parser and the Qwen 3 quarter tool parser, which is exactly what I did. One lesson from my own tests, a short question like that, hash map one doesn’t need thinking at all. The card has an enable thinking switch and for the batch tests I flipped it off. A 400-word story then cost around 600 tokens, not thousands. Use thinking where it earns its keep. The cost about 50 minutes of pod time at $2.49 an hour, so around $2.10, and I terminated the pod. Here’s the pod list, empty. If you’re looking for a cheap and reliable GPUs, I use RunPod, and the

09:57 link is in the description. So, what should I test on this next? A quantized version on a small card or a real coding benchmark? Tell me in the comments. Thanks for watching, and I’ll see you in the next one.