12B Coding Model on an 8GB Laptop GPU? I Tested It

Author: Prompt Engineer 48 · Watch on YouTube

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: 12B Coding Model on an 8GB Laptop GPU? I Tested It · Channel: Prompt Engineer 48 · 07:29 · Watch on YouTube

Topics: AI agents, AI evaluation and benchmarks, Model inference and optimization, AI models, Local and open-source models

Summary

The video tests JetBrains’ Mel 2.1 12B thinking model locally on an 8 GB laptop GPU using Ollama and Q4KM quantization. Although the model does not fit entirely in VRAM, it runs with layers split between GPU and system memory, reaching about 57 tokens per second after warming up. It passes a Python coding test, fixes a broken Snake game after receiving a symptom report, makes a correct tool call, and answers two additional checks correctly. The creator notes that the tests were limited and that JetBrains’ reported benchmarks put a 9B competitor ahead on some tasks.

Key points

Mel 2.1 has 12 billion total parameters, with 2.5 billion active per token across a mixture of 64 experts.
The model card lists a 131,072-token context window and an Apache 2.0 license.
The recommended Q4KM download is 8.1 GB, so it does not fit fully in the laptop’s 8 GB of VRAM; Ollama reports 25% CPU and 75% GPU usage.
After the initial warm-up, measured generation speed stays around 57 tokens per second across the tests.
A Python merge-intervals function passed five assertions, including the case where intervals touch.
The first Snake game draft was broken, but the model fixed it after being told that nothing moved and the arrow keys did nothing.
In a simple tool-use test, the model chose the shell tool and formed the correct wc -l readme.md command.
It correctly identified a nonexistent pandas method and answered the divisibility question with 401.
The creator did not test the full 128K context, run SWE-Bench, or evaluate long agent loops.
JetBrains’ reported results show Mel 2.1 ahead on several coding benchmarks, while a 9B model leads on some other measures, including SWE-bench.

Chapters

Start	Chapter	Summary
00:00	Model overview and reported benchmarks	The video introduces Mel 2.1, explains its mixture-of-experts setup, and compares JetBrains’ reported benchmark results with another model.
02:06	Running Mel 2.1 on an 8 GB laptop	The creator loads the Q4KM model with Ollama, describes its CPU/GPU split, and discusses quantization quality and speed.
03:10	Coding tests and a Snake game repair	The model passes a merge-intervals test, then repairs a broken Snake game after receiving a description of the observed symptoms.
04:44	Tool use, reasoning checks, and measured speed	The creator tests a shell-tool call, a nonexistent pandas method, and a math question, then summarizes generation speed and test limitations.
06:22	Limitations and final assessment	The video lists what was not tested, gives a cautious recommendation for local coding use, and previews a future long-context test.

Quotes

12 billion parameters running on this 8 GB laptop GPU at about 57 tokens a second. — 00:00

the layers split between my GPU and my system memory — 02:39

you can see all pass. — 03:40

That’s one clean call, so it’s a good sign. — 04:44

I did not run long agent loops. — 06:22

Concepts and entities

Mellum 2.1, JetBrains, Ollama, Q4_K_M, GGUF, Qwen3.5 9B, SWE-bench, LiveCodeBench, Apache License 2.0

Full transcript (original language)

00:00 12 billion parameters running on this 8 GB laptop GPU at about 57 tokens a second. This is JetBrains new Mel 2.1 thinking model, and you can see it reasoning right here on my local system. Now, mind you, nothing here is on a rented GPU. Let’s go ahead and test this out, and I’ll show you where it breaks to. I’m prompt engineer, and I tested this on my own laptop. So, everything you see next is real. All right, this is the model card on Hugging Face. It’s Mel 2.1 12 billion parameters in total, but is a

00:32 mixture of experts, so only 2.5 billion are active for each token. If you look here, there are 64 experts, and eight of them are activated per token. The context window is 131,072 tokens, and the license is Apache 2.0, so you can use it commercially. Now, JetBrains says the architecture did not change from Mel 2. What changed is the post-training. Most of the work went into reinforcement learning in real repositories with a shell and file tools, and the reward is when the tests pass. Okay, so here are the numbers

01:03 JetBrains reports, and mind you, these are self-reported on SWE-bench. Verified, Mel 2.1 gets 47.0, and the old Mel 2 got 2.0. LiveCodeBench version 6 is 82.0, AIME is 83.3, and TerminalBench 2.1 is 17.4. But, if you look at the column on the right, Qwen 3.59B is ahead on several of these. It has 50.0 on SWE-bench. Verified, 86.7 on AIME, and 77.8 on GPQA Diamond, where Mel has

01:34 64.6. Where Mel wins is the coding block LiveCodeBench 82.0 against 75.4, HumanEval+ 91.5 against 89.6, and MBPP+ 79.4 against 69.8. It also wins on BFCL version 4 tool use 62.3 against 58.5, and it beats Gemma 4E4B on every coding row in this table. So, the honest read is this, it’s a close second to a 9B model on most of these with fewer active parameters and it’s open under Apache

02:06 2.0. And the agentic numbers were run in JetBrains’ own harness, so I can’t confirm those on my laptop. What I can test is whether it runs and whether it it gets things right. Hold on to the speed number because there’s a surprise about how it fits in 8 GB. Now, for running it locally, JetBrains has a GBUF repo and these are the sizes. BF16 is 24.3 GB, Q8 is 12.9 and the Q4KM they recommend is 8.1 GB. My card has 8 GB of VRAM and three of those were already

02:39 used by other apps, so it won’t fit fully. What I’m going to do is pull the Q4KM with Ollama one command and go ahead and run it. Once it loads, Ollama PS says 25% CPU and 75% GPU. So, the layers split between my GPU and my system memory and because only 2.5 billion parameters are active per token, it still keeps up. First load took about 32 seconds. The GGOF page also lists how much quality each quant keeps against the BF16 original. Q4KM has a KL

03:10 divergence of 0.075 and matches the top token 88.0% of the time while Q8 is at 96.1% and the MXFP41 is at 85.6. So, Q4KM is a real trade, not free. The one thing to mind here, that first prompt was slow about 36 seconds because the model was still warming up. After that, it settles to about 57 tokens a second. First test is plain code. I asked for a Python function merge intervals where touching

03:40 intervals like one to three and three to five have to merge. It thought for a bit and wrote the function 615 tokens in about 26 seconds and then I did not just read it. I ran it against five asserts including the empty list and the touching case and you can see all pass. So, that’s a good start for the coding side. Next, I asked for a full snake game in a single HTML file, canvas, arrow keys, score, game over, restart on space, and speed going up with the score. It wrote 3,147 tokens in about 56 seconds and the file

04:11 is only about 3.5 kilobytes. I opened it in Chrome and the canvas draws and the score shows but nothing moves. So, this first draft is broken. I read the code and the bug is one line. The snake starts with direction zero, so the very first move counts as the snake hitting itself and it’s game over before you press anything. So, I told it only what I saw, “Nothing moves and the arrow keys do nothing.” It thought for about 24,000 characters, 7,700 tokens in 158 seconds and it changed that one line. So, the snake starts moving right. Then, I

04:44 played the fixed version with a small script that presses the real arrow keys and it scored 12 in 24 seconds without dying. So, it was broken on the first try and fixed on the second from a symptom alone. This one matters for agents. I gave it two tools, run shell and read file, and asked how many lines are in readme.md. It picked run shell and send the command wc -l readme.md with the right argument shape. That took about 10 seconds. That’s one clean call, so it’s a good sign. But, it’s one call. I did not run a long agent loop, so I can’t tell you how it does across 50 steps in a real

05:18 repo. Now, two quick checks. First, a trap. I asked what pandas dataframe from Nike returns, a method that does not exist. It said the method does not exist and that calling it raises an attribute error instead of making up a return type. That’s the answer I wanted. Then, a math one with a known answer, “How many integers up to 1,000 are divisible by three or five but not 15?” The answer is 401 and it got 401 in about 15 seconds and 838 tokens. One thing about

05:48 this model is that it always thinks first and you can read all of it. On the math puzzle it wrote about 2,200 characters of thinking and the answer itself was three characters 401. On the snake game it wrote about 6,000 characters of thinking and then about three and a half thousand characters of code. So good part of those 57 tokens a second is thinking and not the answer and that’s why the snake game took 56 seconds. Now the speed stayed almost flat across all five tests. I measured 55.9, 57.0, 57.1, 57.1 and 57.1 tokens a

06:22 second. So it did not slow down on outputs up to about 3,000 tokens. The repair run with a longer conversation came in at 54. Past that I did not measure. So here’s what I did not test. I did not push the 128K context because at that size the memory use is going to be a lot higher than my 80 gigabytes can take. I did not run SWE-Bench myself and I did not run long agent loops. If you have an 8 gigabyte laptop and you want a local coding model that thinks this runs. If you need top agent scores, JetBrains own table says a 9B QN is

06:58 still a bit ahead on SWE-Bench. If you’re looking for cheap and reliable GPUs, I use RunPod and the link is in the description. So in this video we saw how to run Melom 2.1 locally on an 8 gigabyte laptop with Ollama and what it did on code, tools and reasoning. If you’re interested go ahead and try the Q4KM on your own system and let me know how it goes. Model used JetBrains Melom 2.1 12B A2.5B thinking. Next up I’ll push a long

07:29 context test so check out the next video. If you want more honest local model tests like this one, subscribe and I’ll see you in the next video.