AI No Longer Needs Backpropagation

Author: Better Stack · Watch on YouTube

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: AI No Longer Needs Backpropagation · Channel: Better Stack · 02:35 · Watch on YouTube

Topics: Model inference and optimization, AI models

Summary

Q Labs introduced Dust, a method for training transformer models without using backpropagation to estimate gradients. Instead of perturbing weights, Dust adds random noise to layer activations and uses token-level loss changes as parallel experiments to estimate gradients. In tests on small GPT-style models, Dust outperformed backpropagation on shorter training runs and substantially outperformed the guessing method Egg Roll, though it did not catch up to backpropagation on longer runs. Dust currently requires much more compute, but avoiding the need for differentiable models could enable new architectures.

Key points

Backpropagation estimates gradients by sending information backward through a network using the chain rule.
Backpropagation requires every part of a model to be differentiable.
Evolution strategies estimate useful directions by randomly nudging weights and checking how the loss changes.
Dust perturbs activations rather than weights, adding independent noise to each layer’s output for every token.
Dust treats the tokens in a sequence as a virtual population of parallel experiments within one forward pass.
The method uses changes in loss to estimate gradients, then computes weight updates in the same way as backpropagation.
In tests on small GPT-style models, Dust did better than backpropagation on shorter training runs but did not catch up on longer ones.
Dust outperformed Egg Roll even when Egg Roll was allowed 256 times more guesses.
Dust needs orders of magnitude greater efficiency to become practical, and the largest tested model had 243 million parameters.
Training without requiring differentiability could make architectures involving external programs or repeated self-loops more feasible.

Chapters

Start	Chapter	Summary
00:00	How backpropagation works	The video explains how backpropagation calculates gradients and why its requirement for differentiable components constrains model design.
01:01	Dust estimates gradients through activation noise	Dust perturbs layer activations and uses token-level loss changes to form a virtual population of experiments and estimate gradients.
01:33	Dust’s results in model tests	Tests on small GPT-style models show promising results against backpropagation on shorter runs and against the guessing method Egg Roll.
02:05	Compute costs and future possibilities	Dust remains too compute-intensive for practical use, but removing the need for differentiability could open up new model architectures.

Quotes

Every single piece of your model has to be differentiable — 00:31

Q Labs’ method is called Dust, and it’s a trick to stop nudging the weights and nudge the activations instead. — 01:01

The only difference is that the gradient comes from trial and error instead of from calculus. — 01:33

Dust needs a lot more compute than backpropagation — 02:05

your model doesn’t need to be differentiable at all. — 02:35

Concepts and entities

backpropagation, evolution strategies, Dust, Egg Roll, Q Labs, Transformer, GPT, GitHub

Full transcript (original language)

00:00 Every AI model you’ve ever used was trained with the same algorithm called backpropagation, which was popularized all the way back in 1986. But a team at Q Labs just pre-trained a transformer without it, and in some of their tests, their method actually beat backpropagation. Let me explain how they did it. So, when you train a neural network, you run data forward through a model and measure how wrong the output is, which is called the loss. Then, you need to figure out which way to adjust each of the model’s weights so the loss goes down, and that direction is called the gradient. Backpropagation finds the

00:31 gradient by running a second pass backwards through every layer using the chain rule from calculus to work out exactly how much each weight contributed to the error. It’s precise and efficient, but it comes with a big requirement. Every single piece of your model has to be differentiable, and that constraint has shaped pretty much every architecture, optimizer, and GPU we use today. The alternative has always been search. Instead of calculating the gradient, you just guess. You nudge the weights randomly, run the model, and check whether the loss went up or down. If you do that across a big population

01:01 of random nudges, the good directions start to add up. These are called evolution strategies, and the problem is that they’re painfully slow because every guess needs its own copy of the weights and its own forward pass. Q Labs’ method is called Dust, and it’s a trick to stop nudging the weights and nudge the activations instead. It adds a tiny bit of random noise to the output of each layer independently every single token and then checks how the loss changed at each token. So, one sequence of 2,048 tokens becomes 2,048 separate experiments running in parallel inside a single forward pass. The researchers

01:33 called this a virtual population. The noise that lowered the loss gets rewarded, everything gets averaged out, and you end up with an estimate of the gradient at each layer. From there, the weight update is computed exactly the way backpropagation does it. The only difference is that the gradient comes from trial and error instead of from calculus. And the results are quite surprising. They tested it on small GPT-style models, and on the shorter training runs, Dust actually ended up with a smarter model than backpropagation did. On the longer runs, it didn’t quite catch up, but the more guesses they gave, the closer it got.

02:05 They also compared it to the best existing guessing method called Egg Roll, and the difference was huge. Even when Egg Roll was allowed 256 times more guesses, it still couldn’t match Dust. And bigger models actually got better at learning this way, which is the opposite of what everyone expected. Now, there is a big catch though. Dust needs a lot more compute than backpropagation, and the authors say outright that it has to get orders of magnitude more efficient before it’s practical. The largest model they tested was only 243 million parameters, but the long-term idea is really interesting because if you don’t

02:35 need a backward pass, your model doesn’t need to be differentiable at all. That opens the door to architectures that backpropagation struggle with, like models that call external programs mid-computation or loop over themselves many times. The code is already on GitHub if you want to dig deeper into it, and if you want to stay up-to-date with the latest in AI research, be sure to subscribe to the Better Stack channel.