I Ran Stanford's "LLM Killer" on My Laptop

Author: Hyperautomation Labs · Watch on YouTube

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: I Ran Stanford’s “LLM Killer” on My Laptop · Channel: Hyperautomation Labs · 05:11 · Watch on YouTube

Topics: AI agents, AI evaluation and benchmarks, AI content generation, AI models, Local and open-source models

Summary

The video compares conventional word-by-word language models with Stanford’s method, which drafts an entire answer in layers and can revise earlier mistakes. The creator runs Stanford’s hard-Sudoku model on a Mac laptop and solves 1,726 of 2,000 boards, closely matching the reported result. The video notes limitations: the models are small, the speed comparison is not against ChatGPT, and it is unknown whether the results will hold at larger scales. It also describes a separate approach that lets reasoning models think in internal numbers, potentially using fewer tokens, while concluding that these research ideas are promising but do not yet make an “LLM killer.”

Key points

Conventional chatbots generate text piece by piece, with each new piece requiring another model pass.
Stanford’s approach drafts a whole answer in layers, allowing later passes to revise earlier parts.
On hard Sudoku, the word-by-word test model solved about 1% of boards in 128 passes; Stanford’s model solved 86% in 12 passes.
The creator adapted the model to run locally on a Mac, without cloud computing.
On 2,000 hard Sudoku boards, the model solved 1,726, or 86.3%, in about 50 seconds.
A sample board was solved in 12 passes, though another finished with six incorrect cells.
The cited 31-times-faster result compares the model with another experimental diffusion model, not with ChatGPT.
The largest NCM described has about 700 million parameters, and it is not yet known whether the approach scales to the size of leading open chat models.
A separate hybrid-agent approach uses internal numbers for reasoning and switches to words for tool use or final answers; its authors report up to 55% fewer tokens and up to 3.7 times lower cost.

Chapters

Start	Chapter	Summary
00:00	Why word-by-word generation can struggle	The video introduces Stanford’s alternative to sequential text generation and explains why Sudoku tests the ability to revise an answer.
01:33	Layered answers and a laptop test	The model builds a rough answer and refines it in layers, then the creator runs Stanford’s Sudoku test set on a Mac.
03:06	Results and limitations	The video shows a solved board, explains what failures look like, and clarifies the limits of the speed and model-size comparisons.
04:39	A second approach and the verdict	The video describes hybrid agents that reason in internal numbers and concludes that the ideas are promising but not yet an LLM killer.

Quotes

It sketches the whole answer first, then sharpens it. — 00:00

No cloud, just this laptop. — 02:04

50 seconds later, 1,726 solved. — 02:35

It isn’t filling in one cell at a time. It’s reworking the whole board at once. — 03:06

So, is it the LLM killer? Not yet. — 04:39

Concepts and entities

Stanford, ChatGPT, NCM, Sudoku, diffusion, GitHub, NVIDIA, Hybrid agents, Together AI

Full transcript (original language)

00:00 Ask Chat GPT a hard question and watch how the answer shows up. One word, then the next, then the next. Once a word is on the screen, it can’t go back and change it. If word three is wrong, everything after it is built on that mistake. This month, Stanford released an AI that doesn’t write that way. It sketches the whole answer first, then sharpens it. On hard Sudoku puzzles, a word-by-word model solved one out of a hundred.

00:31 Stanford’s model the same size solved 86. People are calling it the LLM killer. Every video I found explained the paper. None of them ran it. So, I downloaded it and ran it on this laptop. Almost every chatbot you use works like this. It predicts one small piece of a word, adds it, then runs the whole model again for the next piece. A long answer means hundreds of scripts through the model, one after another.

01:02 Sudoku shows why that’s a problem. Every number depends on numbers you haven’t written yet. Fill it in left to right, and a mistake in the first row only shows up five rows later when it’s too late to fix. That’s the test Stanford picked. A small word-by-word model got 128 passes and solved about 1% of the hard boards. Next concept. Modeling flips the order. First, it learns to squeeze a whole answer into stack of codes. The top of

01:33 the stack is the rough idea. Each layer below adds more detail. When it writes, it fills in a whole layer at once. Layer one is a blurry guess at the entire answer. Layer two sharpens all of it. By the last layer, every piece is in place. Because every pass works on the whole answer, a later layer can fix a mistake an earlier one made. On Sudoku, that took 12 passes instead of 128.

02:04 The code and the models are free on GitHub. The hard Sudoku model is about 160 megabytes. That’s smaller than most phone games. It’s built for Nvidia graphics cards, so I changed two small things to run it on a Mac. No cloud, just this laptop. I gave it all 2,000 hard boards from Stanford’s test set. 50 seconds later, 1,726 solved.

02:35 86.3%. Stanford reported 86.1. That’s about a 40th of a second per board. Now, let’s slow one down. 30 clues, 51 empty cells. After the first pass, only 15 cells are right. It’s mostly guessing. Between pass four and pass five, the score even goes down from 21 to 19. It isn’t filling in one cell at a time. It’s reworking the whole board at once. Pass eight, 42 right. Pass 12, 81 out of

03:06 81. Solved in about a 10th of a second. It doesn’t always win. On this board, it finished with six cells wrong, so it doesn’t count. That’s what the other 14% look like. Now the fine print. You’ll see a 31 times faster number. That’s real, but it’s compared with another kind of experimental model called diffusion, not with chat GPT. The 1% model was also a tiny test model,

03:37 the same size as Stanford’s. The biggest NCM they trained is about 700 million parameters. Top open chat models have hundreds of billions. Nobody knows yet if this holds up at that size. It does write real sentences, though. Here’s a German news line it translated next to a human translation. Close. A second paper from Stanford, together AI and others, goes after the same habit from the other side. When a reasoning model thinks, it

04:07 thinks in words you never see. And you still pay for those words. Hybrid agents lets the model think in its own internal numbers instead of words and switch back to words only when it uses a tool or gives you the answer. By the authors’ numbers, that used up to 55% fewer tokens and cost up to 3.7 times less. The obvious worry, if it thinks without words, how do you check what it was thinking? The team built a viewer that turns those hidden

04:39 thoughts back into rough text you can read. So, is it the LLM killer? Not yet. These are small research models tested on puzzles, summaries, and translation. But the idea is real and it runs on a laptop today. Sketch the whole answer, then fix it instead of writing word by word and hoping. If the big labs scale it up, the AI you use could stop locking in its own mistakes. I put the exact steps I used into a free

05:11 guide. The commands, the two changes for the Mac, and both papers in plain English. Comment Sudoku and I’ll send it to you.