Germany's New 78B Open Model (Kolibri-1) Runs at 126 Tokens/Sec on ONE GPU (I Tested It Hard)

Author: Prompt Engineer 48 · Watch on YouTube

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: Germany’s New 78B Open Model (Kolibri-1) Runs at 126 Tokens/Sec on ONE GPU (I Tested It Hard) · Channel: Prompt Engineer 48 · 15:39 · Watch on YouTube

Topics: AI evaluation and benchmarks, Prompt and context engineering, Model inference and optimization, Local and open-source models

Summary

The video tests Aleph Alpha’s Colibri 1, a 78.1-billion-parameter mixture-of-experts model with 3.46 billion active parameters per token, running as a Q4 GGUF on one A100 GPU. It reached 126.45 tokens per second in a benchmark and about 118 tokens per second in server tests, while handling several math, logic, tool-calling, and long-context tasks well. The tests also exposed drawbacks: high reasoning effort can consume large token budgets without producing an answer, visual and coding outputs can be unreliable, and closed-book facts may be wrong. The million-token context worked only after a configuration override, and the video concludes that Colibri 1 may suit users needing German-English long-document processing on their own hardware, but is less suitable as a coding agent or factual-recall model.

Key points

Colibri 1 is an open-weight mixture-of-experts model from Aleph Alpha, with 78.1 billion total parameters and 3.46 billion active per token; its stated license is Apache 2.0.
The model supports German and English, reasoning modes with low, medium, and high effort, and tool calling.
The Q4 GGUF ran on one A100 SXM with 80 GB VRAM, generating at 126.45 tokens per second in Llama bench and about 118 tokens per second in server tests.
German and math examples were answered correctly, but some high-effort reasoning and coding prompts exhausted their token budgets without returning a usable result.
Visual code-generation results varied: the snake game passed, while the solar system and bouncing-ball demos had flaws, the Mandelbrot initially failed, and the pelican drawing did not resemble the requested subject.
The model repaired a bug in its own interval-merging test after feedback and made a clean parallel call to weather and meeting-booking tools.
A context override was required to enable the advertised million-token window; the model retrieved a planted passphrase at about 196,000 and 719,000 tokens.
The model declined to summarize a fabricated research paper, but misstated its own knowledge cutoff and gave incorrect goal scorers for the 1998 World Cup final.
The Q4 weights use about 44 GB, so they barely fit on a 48 GB card at short context; the tested 80 GB card provided more room.

Chapters

Start	Chapter	Summary

Quotes

Generation is 126.45 tokens per second. — 05:40

Give this model a big token budget or lower the reasoning effort to low for creative and coding tasks. — 07:15

The fix on llama.cpp is one flag — 11:29

The answer was indigo pangolin 77 exactly. — 13:01

If you need German and English long documents and you want to run it on your own machine, it is a strong candidate. — 15:08

Concepts and entities

Kolibri-1, Aleph Alpha, GGUF, llama.cpp, RunPod, A100, Hugging Face, needle in a haystack

Full transcript (original language)

00:00 All right. So, a German company just released a 78 billion parameter model and I ran it on 1 A100 at 118 tokens per second. Mind you, only 3.46 billion of those parameters work on each token. That is the whole trick and in this video we are going to test whether the trick holds up. It is called Colibri 1. It is open weight and it came out on the 3rd of October. Now, I already made a separate video where I convert the safe tensors files into GGUF step by step. That one is

00:30 about 7 and 1/2 minutes and the link is in the description. The GUF I made is on Hugging Face under prompt 48 Colibri 1 GGUF. In this video we are going to take the Q4 version from that repo, rent a GPU on RunPod, and try to break it. So, what is Colibri? It is a mixture of experts reasoning model from Aleph Alpha and Colibri is the German word for hummingbird. If you look at the model card here, it has 78.1 billion parameters in total and 3.46 billion active per token. The

01:01 license is Apache 2.0. The languages are German and English and the knowledge cutoff is the 18th of June 2026. It supports a reasoning mode with low, medium, and high effort and it supports tool calling. The company pitch is sovereignty. They say it was built in Germany, trained on infrastructure in Germany and Finland under European law, so hospital or a bank can run it on their own machines with no outside party in control. That is their claim and it is a real selling point for a lot of organizations.

01:32 Now, let’s go ahead and read the config file instead of the marketing page because that is where the interesting part is. There are 50 layers, 40 of them use sliding window attention with a window of 513 tokens in the config and 10 of them use full attention, one in every five layers. The model card says 512 and the config says 513, so I’m going with the config here. Each layer has 384 experts, six routed experts plus one shared expert fire for every token. That is how you get 78

02:03 billion in storage and 3.46 billion in compute. And there are only four key value heads with a head dimension of 128. Put those together and the cache gets small. I worked it out before touching the GPU at 262,000 tokens. A normal dense layout would need 25 GB of cache in 16-bit and this layout needs about five. That is five times smaller and I will check that number later with a real measurement. On training, the report says 20 trillion tokens in pre-training, roughly 62%

02:33 English, 24% German, and 14% code. Then 3.44 trillion tokens in mid-training and about 200 billion for long context. The hardware was 768 B200 GPUs for 21 days. They used the Meon optimizer and a routing method they call exact quantile balancing so that the experts share the work evenly. All of this is from the vendor. I did not train anything so take it as what they report. Now the benchmarks. Everything on this screen is self-reported by Aleph Alpha

03:04 run with their own evaluation framework. They say the overall English score is 75.5 against 74.7 for a Qwen 3.5 35B with 3B active. German is 70.8 against 69.8. AIME 2025 in English is 96.9. GPQA Diamond is 84.3. The same table also has the bad news and credit to them for printing it. Close book retrieval is 51, the last of 12 models according to Tejas Kumar’s write-up. Multi-turn tool calling on

03:34 BFCL is 39.8. SWE-Bench verified is 66.4. Tejas also points out that at 128,000 tokens, the ruler score dips to 67.9 even though the million token score is higher. As for independent numbers, I searched for Colibri on artificial analysis, LM Arena, and Live Bench on the day of release, and I could not find any. So, nobody independent has verified these tables yet. That is why the rest of this video is me testing things myself.

04:04 Here’s the first problem. The official way to run this model is a vLLM plugin from Aleph Alpha, and the minimum hardware on the card is two A180 gig cards or one H200. The weights are about 78 GB in FP8. The card does not mention Llama.cpp or Llama or LM Studio anywhere. A stock build cannot load the architecture. That is why the GGUF I published needs a patched Llama.cpp, and the patch comes from Hob Forge, who did the hard work of adding the architecture. Credit to them,

04:35 it is linked in the repo. My Q4KM file is 47.45 GB. If you want to know how I converted it, that is the other video. So, let’s go ahead and rent a GPU. This is a run pod with one A100 SXM 80 GB of VRAM at $1.59 an hour. I downloaded the Q4 file straight into RAM, cloned Llama.cpp at the same revision as my model card, applied the patch, and started the CUDA build. And you can see it failed. CMake says

05:07 the CUDA compiler is not found. The NVCC binary was on the machine, but it was not on the path. I added it to the path, passed the compiler location to CMake, and the second build went through. My model card says GPU backends are unverified because my own tests in that video were CPU only. So, this is the first time this file runs on a GPU. All right, Llama bench with every layer offloaded to the GPU. The model loads as 44.19 GB, 78.1 billion parameters. Prompt processing is 2,497 tokens per

05:40 second. Generation is 126.45 tokens per second. So, a 78 billion parameter model generates faster than most small dense models do on the same card because only 3.46 billion parameters are working. In the real server with reasoning on, I saw about 118 tokens per second across my tests. That part of the promise is real on this card. Let’s start with simple things. First, a German question. Why Germany was organized as a federation after 1945?

06:12 Answer in five sentences and name two articles of the basic law. You can see that the German is fluent and it names article 20 and article 30 and both of those exist and say what it claims. Next, math. How many ordered pairs of numbers from 1 to 100 have a square sum divisible by 7? I brute-forced this on my laptop first and the answer is 196. The model reasoned through the squares modulo 7, worked out that both numbers must be multiples of 7 and answered 196.

06:43 About 3,200 tokens, 27 seconds. Then, a seating puzzle with five people. My brute force says 10 arrangements. This is where I hit the first problem with the model. With high reasoning effort and a 12,000 token limit, the model spent the whole budget thinking. The reasoning trace was almost 35,000 characters and the final answer was empty. I raised the limit to 40,000 tokens and ran it again. It answered 10, which is correct after 12,400 tokens and about

07:15 106 seconds. The same thing happened with the coding tasks. I asked for a snake game with medium effort and 40,000 tokens and I got 53,000 characters of thinking and no game. So, the practical advice is simple. Give this model a big token budget or lower the reasoning effort to low for creative and coding tasks. This is a pattern I want you to remember because it comes back. Now, the visual part. Colibri is a text model. It cannot look at images, so the way I test visuals is by asking it to

07:45 write code that draws things and then I open the result in a browser. Snake first with low effort, about 9,700 tokens. I wrote a small bot that plays it by reading the game state. You can see the score climbing to 20, the speed going from 150 milliseconds per step down to 50, and the high score updates. No errors in the console. This one is a pass. Second, an animated solar system, 3,100

08:17 tokens. It renders, the sun glows, planets orbit, but if you look carefully, there are huge stray ellipse arcs crossing the screen. The slider is clipped at the bottom, and not all eight planets are in the frame. It works, it is not pretty. Third, 20 balls bouncing inside a rotating hexagon. It runs, gravity works, balls collide with each other and the walls. The numbers on the balls are missing and the hexagon outline is

08:48 barely visible. A partial pass. Fourth, a Mandelbrot explorer. This one cost 22,700 tokens, nearly 3 minutes, and the canvas is completely black even at zoom one. That is a fail. And last, the pelican riding a bicycle. At medium effort, it drew a blob on two wheels. With high effort and 5,600

09:19 tokens, it drew a snowman shape sitting on a barbell, no bicycle. The art is a fail. Now, the part I like. I asked for a Python function that merges intervals with a self-test block. The model wrote it, I ran it, and it crashed on its own assert. If you read the assert, the bug is in the test. A reverse pair gets normalized, so the merge should give one interval and the model’s own test expected two. I pasted the error back and said, “Fix whatever is wrong.” One round, about 63

09:51 seconds, and the output says all tests pass. That is the loop I would actually use this model for. I tried the same loop on the Mandelbrot. I told it the canvas is black at zoom one and nothing else. About 16,000 tokens and 4 minutes later, it gave me a version that draws the full fractal at zoom one. The zoom still drifts after four clicks on the same spot. The view ends up in the corner of the screen. So, I would call that half fixed.

10:24 Tool calling next. Because this is where the benchmark table was weakest. I gave it two tools, a weather lookup and a meeting booker that wants time in 24-hour format. Then I wrote, “Book the meeting at 4:00 p.m.” You can see it called both tools in parallel in one turn. The weather call has Heidelberg and Celsius, and the meeting call has the date and time 16:00. Nobody told it to convert 4:00 p.m. It did that on its own. I fed fake results back, 14° with light

10:57 rain, and the second turn composed the answer using only those fields, including the meeting ID, no invented data. One test is not a benchmark, so I’m not going to claim 39.8 is wrong. I’m saying this one was clean. Now, context length, the headline feature, 1 million tokens, says the card. I started the server with a context of 1 million, and the log said the slot context exceeds the training context of the model and is capping. The slot was 262,144 tokens, not a million.

11:29 I dumped the GG GUF metadata and found the reason. Context length is stored as 262,144. That value came from max position embeddings in the original config. All of Alpha’s own instructions for VL LM say you need an override to get the million. So, this is a thing we have to fix in my GGUF model card and I will add the override instructions there. The fix on llama.cpp is one flag, override kv colibri 1.context_length set to 1,048,576.

12:01 After that, the log says slot context 1,048,576, no capping. Here’s the check I promised. I loaded the server at different context sizes and read the VRAM. At 8,000 context, 45,929 MB. At 262,000, 51,137. At 1,000,067,267. The difference between 8,000 and 262,000 is 5.09 GB. My prediction from the

12:31 config was 4.84. That is within about 5%. So, a full million tokens of cache fits on 1A 100E with a Q4 weights. Fitting is different from working, so the next test is the needle. For the needle in a haystack, I build a long text of boring repeated sentences and plant one line at 46% depth. The line says the secret vault passphrase is indigo pangolin 77. Then I ask for the passphrase at the end. At about 196,000 tokens, it took 122

13:01 seconds and the answer was exactly indigo pangolin 77. Only 323 characters of reasoning. Then the big one with the override on, about 719,000 tokens, same passphrase at 46% depth. It took 914 seconds, about 15 minutes, and generation slowed to 50 tokens per second at that depth. The answer was indigo pangolin 77 exactly. The card says 1 million tokens. I tested 719,000 and it found the needle.

13:34 Two honesty tests. First, a fake paper. I asked it to summarize a NeurIPS 2023 paper that I made up with invented authors. It did not invent a summary. It said it did not have the paper and that it might be hypothetical. Good. But in the same answer it said its knowledge cutoff is October 2023. The model card says June 2026, so it does not know its own cutoff. Second, closed book trivia. Who won the 1998 World Cup final? France 3-0 against Brazil, correct. Capital of Burkina Faso

14:07 and independence year, correct. But the goal scorers were wrong. It said Zidane scored a penalty and that Blanc scored. In the real match, Zidane scored two headers and Petit scored the third and Blanc did not score in the final. That matches what the vendor table says about closed book knowledge. With a model like this, give it documents to read. Do not ask it to remember facts. So, what is the verdict? Let me put it as a list. What worked? 126 tokens per second at Q4

14:37 on 1A 100, correct. Math and logic, clean parallel tool calling with argument conversion, a self-repair loop that fixed its own bug in one round, a perfect needle at 196,000 tokens, and again at 719,000 with the override. What did not work? It overthinks and burns the whole budget on medium and high effort. Two of the five visual tests failed outright and two more were only partial. Closed book facts are unreliable and the million token window is switched off in the GUF until you override it.

15:08 About the GPU, the Q4 file is 44 GB bytes of weights. On a 48 GB card, it barely fits at a short context. I measured 45.9 GB in use at 8,000 tokens. An 80 GB card is comfortable. Anything smaller needs offloading to system memory and I did not test that so I cannot tell you the speed. Who is this for? If you need German and English long documents and you want to run it on your own machine, it is a strong candidate. If you want a coding agent or a model that remembers facts, I would look elsewhere for now.

15:39 So in this video we have seen Colibri 1 running on a real GPU with a speed test, a context test and a set of tough examples including the ones that failed. If you want to see how I made the GBUF, watch the conversion video. I show it step by step from safe tensors to GBUF. And if you’re interested in reading further, I will link to the main page in the description below. Let me know how it goes if you try it on your own system. I will see you in another one. Check out the next video.