Strata: Run a 125B AI Model on a Cheap 12GB GPU (Free)

Author: Hyperautomation Labs · Watch on YouTube

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: Strata: Run a 125B AI Model on a Cheap 12GB GPU (Free) · Channel: Hyperautomation Labs · 05:48 · Watch on YouTube

Topics: AI agents, Model inference and optimization, AI infrastructure and hardware, Local and open-source models

Summary

The video introduces Strata, a free project that runs the 125-billion-parameter Qwen 3.8 Flash Next model on a PC with a 12 GB graphics card by distributing work across the GPU, RAM, processor, and SSD. It explains the model’s expert-based design and speculative decoding, then reports performance figures measured by the Strata team on different graphics cards. The video outlines hardware requirements, model choices based on available RAM, and installation steps. It also explains how to connect Strata to coding tools for local use, while noting limitations such as long initial prompts and weaker performance on the hardest reasoning tasks.

Key points

Strata is presented as a free way to run a 125-billion-parameter model on a 12 GB GPU.
The model uses a mixture-of-experts design: only a subset of its 24,576 experts is used for each word.
Strata keeps frequently used experts on the GPU while using RAM, the processor, and SSD storage for other parts of the workload.
A small helper model proposes words that the larger model checks, which the video says can make generation 1.6 to 1.8 times faster.
The team reports up to 94 tokens per second on an RTX 5070, depending on model size.
The stated minimum requirements include a supported recent Nvidia or AMD GPU with at least 12 GB of memory, 32 GB of RAM, and about 80 GB of free SSD space.
The installer can check the PC and choose a model; the video also describes a manual setup option for Windows and Linux.
Strata can be connected to compatible coding tools so requests run locally, without per-token API charges or rate limits, according to the video.
The video notes that the first message in a long chat can take about a minute per 30,000 tokens, and that top paid models still lead on the hardest reasoning tasks.

Chapters

Start	Chapter	Summary
00:00	Introducing Strata and the model	The video presents Strata as a way to run a 125-billion-parameter model on a 12 GB graphics card and describes the model’s claimed capabilities.
01:36	How Strata fits the model on a PC	The video explains how Strata distributes model work across GPU memory, system RAM, the processor, and SSD storage.
02:39	Speed and benchmark figures	The video describes speculative decoding and shares the Strata team’s reported token-generation speeds on Nvidia and AMD graphics cards.
03:42	Requirements, model choice, and installation	The video lists hardware requirements, recommends model sizes based on RAM, and walks through the installer and first launch.
04:47	Connecting coding tools and limitations	The video explains how to use Strata with coding tools and summarizes local-use benefits alongside performance caveats.

Quotes

This free project runs a 125 billion parameter model on a 12 gig card. — 00:00

Strata doesn’t try to squeeze the model onto your card. It spreads it across your whole PC. — 01:36

The big model still decides every word, so you get the same answer, just 1.6 to 1.8 times sooner. — 02:39

The first start can freeze your PC for 1 to 3 minutes while it loads the model. — 04:47

nothing you write ever leaves your machine. — 05:17

Concepts and entities

Strata, Qwen3.8 Flash Next, Mixture of Experts, speculative decoding, Claude Opus 4.6, Claude Code, Cursor, Codex, RTX 5070, OpenAI, Anthropic

Full transcript (original language)

00:00 You’ve got a gaming PC, a graphics card with 12 GB on it, and every month you still pay for an AI that runs on somebody else’s computer. Because the really good models, the ones that write real code, need a server. Hundreds of GB of graphics memory. Your card has 12. Well, 3 weeks ago, that stopped being true. This free project runs a 125 billion parameter model on a 12 gig card. 22,000 people started in 18 days.

00:35 It’s called Strata. In the [snorts] next 7 minutes, how a model this big fits in a card this small, how fast it really is, whether your PC can run it, and how to plug it into cloud code so your coding agent runs free and private on your own machine. The model is Quen 3.8 flash next. 125 billion parameters. But here’s the trick. For each word, only 6 billion of them actually work. And it’s good. On Quen’s own published tests, it

01:05 beats Claude Opus 4.6 on real world coding, edges it on competitive coding and science questions, and loses on the hardest reasoning exam. These are Quen’s numbers for the full model, so treat them as the ceiling. The point is, this is not a toy. This is the class of model you pay a subscription for. Here’s the problem. At full size, this model is about 250 GB. Compressed, the download is still around 70. Your card holds 12, six times too big. Every other tool hits

01:36 this wall and crawls. Strata doesn’t try to squeeze the model onto your card. It spreads it across your whole PC. Inside the model is a team of 24,576 small specialists called experts. Each word only needs 10 of them. So Strata keeps the few thousand experts that get used most right on your graphics card and it keeps learning which ones those are while you chat. Your RAM holds all of them. When a word needs an expert, the card doesn’t have. Your processor

02:07 works on it at the same time as the graphics card, so neither one waits. and your SSD holds a big lookup table. The model reads just a few rows of it per word. That’s the whole idea. Your graphics card, RAM, processor, and SSD each do one job. And every extra gigabyte on your card holds about 700 more experts, which is why a bigger card is faster. Then there’s a speed trick. A small helper inside the model guesses the next few words. The big model checks

02:39 all the guesses in one go. keeps the right ones and writes the next word itself. The big model still decides every word, so you get the same answer, just 1.6 to 1.8 times sooner. So, how fast is it? The Strata team measured it on an RTX 5070. A normal 12 gig gaming card with 64 gigs of RAM. The fastest size writes 94 tokens a second. The recommended one 79. The smartest one 53. Anything over 60 is faster than you can

03:10 read. On AMD, an RX970 XT does 60 and it reads what you send at over 2,000 tokens a second. So, a long document goes in in about 15 seconds. A card with more memory goes faster. They estimate a 24 gig 3090 at 100 to 110. Quick one. I put the fit chart, the setup lines, and the exact settings for cloud code, cursor, and codeex on one page. Comment the word strata and I’ll send it to you. Real talk, 90% of you watching haven’t

03:42 subscribed. On Facebook, subscribe is the supporter button. That is what keeps the lights on here. On YouTube, it’s free and it tells the algorithm to send you more. Subscribe to support us. Here’s the part people skip. Will it run on yours? You need an Nvidia RTX20 series or newer or a recent AMD Radian with 12 gigs or more, Windows or Linux, and about 80 gigs free on an SSD. Then your RAM picks the model. 32 gigs, take the coder version. It’s built for code and weaker at everything else. 48,

04:15 IQ2XS, 64, every size fits, and IQ2XS is the one they recommend. 96 or more, take IQ3S, which matches the full model on the published tests. Not sure? The installer checks your PC and picks for you. Installing it is the easy part. If you use claw code, cursor or codecs, paste this one line, and your AI checks your graphics card, RAM, and disk, picks the model that fits, installs it, and starts it or do it yourself. On Windows,

04:47 doubleclick start here. On Linux, run setup. Press enter for the recommended answers. It downloads about 70 gigs and if that stops, run it again and it picks up where it left off. One warning, the first start can freeze your PC for 1 to 3 minutes while it loads the model. That’s normal. Don’t close the window. Then your browser opens Strata with a chat and a live monitor of your hardware. And here’s why this matters for us. Strata speaks the same language as OpenAI and Anthropic. So point cloud

05:17 code at your own PC with one setting and your coding agent runs on this model. Cursor codeex and any open AAI style app work the same way. No API build per token, no rate limits and nothing you write ever leaves your machine. Four things to know. First, you need at least 32 gigs of RAM. By default, it answers one request at a time. The first message in a long chat takes about a minute per 30,000 tokens. Then follow-ups start in seconds. And it’s a compressed model. So, for the very hardest reasoning, the top paid models still lead for everyday

05:48 coding on hardware you already own, though. Nothing else this big runs this fast on a 12 gig card. Comment the word strata and I’ll send you the fit sheet. If you want the full claude code playbook, my paid guides are linked below. Follow along on Instagram and Facebook for byte-size AI and the deep dive playbooks live here on YouTube. Subscribe so you don’t miss one. See you in the next one.