Google is SO back...

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: Google is SO back… · Channel: Wes Roth · 14:25 · Watch on YouTube

Topics: AI agents, Enterprise AI applications, AI models, Robotics and autonomous systems

Summary

The video explains Google’s Dream RSI research, which uses AI agents to simulate research decisions against a record of past experiments and discoveries. Rather than spending compute on executing every proposed path, agents can test exploration strategies in the simulated history and deploy the most promising one. The approach is presented as a way to avoid known dead ends, find more efficient research directions, and improve how experiments are selected. The host sees it as a potentially foundational step toward systems that can conduct experiments and improve the tools and processes they use.

Key points

Research progress can be viewed as a branching tech tree, where choosing the wrong path can waste substantial time and resources.
Past experiments provide data about which research directions succeeded, which failed, and what discoveries depended on earlier work.
Dream RSI uses that history as a simulator in which candidate exploration policies can be evaluated without executing new experiments.
The system aims to learn which experiments to pursue, which branches to abandon, and how to prioritize work.
A fixed exploration strategy cannot learn from accumulated experience and may keep spending resources on directions that have already failed.
The video reports that Dream RSI performed strongly in comparisons, with Gemini RSI using Gemini 3.7 Flash achieving the best score among the approaches discussed.
The host contrasts Dream RSI with AlphaEvolve, describing them as different tools that can help optimize different tasks.
The broader vision is for AI systems to select and run experiments, improve research processes, and help create better future models.

Chapters

Start	Chapter	Summary
00:00	Research as a branching tech tree	The host introduces recursive self-improvement and explains how research choices can lead to productive discoveries or dead ends.
03:13	Using research history as a simulator	Dream RSI treats records of past experiments as a world in which AI agents can test different research strategies.
06:25	Why exploration is difficult	The video discusses the cost and delay of evaluating strategies through real experiments, and how historical data can help avoid repeating mistakes.
10:07	Results and comparison with other approaches	The host reviews reported performance results and describes how Dream RSI and AlphaEvolve can serve different optimization tasks.
12:47	A step toward recursive self-improvement	The video outlines a vision of AI systems that select experiments, improve research processes, and contribute to future models.

Quotes

History is the world to dream in. — 03:44

Dreaming is how it self-improves. — 05:20

A fixed strategy cannot learn from the experiences it accumulates. — 06:57

History is already a simulator. — 09:03

Concepts and entities

Dream RSI, RSI, Google, Gemini 3.7 Flash, AlphaEvolve, Borg, Wes Roth

Full transcript (original language)

00:00 Google dreams of RSI, recursive self-improvement. While most of the country is debating whether or not we should have RSI, Google is like, “Hey, we can put this AI into a simulation and have it do RSI.” Which, I got to say, you got to give them credit for this. So, in this video, let’s take a look at this paper cuz this is kind of a big deal. It’s called Dream RSI. Dream as in simulation. We’re having the AI sort of a dream and run through many, many simulations to figure out how to best improve itself, which pathways to take that lead us there the fastest. And specifically, we’re talking about just

00:34 one aspect of AI research, AI development. So, we can think of it as, let’s say there are two people in the research lab. One of them runs the actual experiments. They’re the hands-on scientists. They’re experimenting and seeing what works. And let’s say the other scientist is the person deciding what we do next. What deserves our attention for the next hour of work. And of course, you can be an absolutely excellent researcher and still waste an entire week of work because you decided to pursue the wrong sort of chain, the wrong branch. Video games have this awesome concept called a tech tree. It

01:07 just means you have to unlock one thing before you unlock another. Right? It’s like Minecraft. You got to build wooden tools, then stone tools, then I don’t know what, eventually diamond tools. I don’t play Minecraft that much, but you get the idea. That’s similar how our discovery system worked, right? We had stone tools, flake tools, honey apparently around the same time, and then eventually got better and better and better and better. Eventually, we had wikis and DVDs and Wi-Fi and GPUs. And apparently, this whole thing ends with AI coding agent in 2025. It’s LM chatbot reasoning model AI coding agent

01:38 and that’s it. End of the line. Why are we talking about this? Because a lot of these scientific research is very much like this. There’s a certain trees that we do go down or branches. Some of them might seem exciting at first, but then lead us nowhere. Whereas, other ones can be almost accidental, but we realize that they’re they’re huge leaps of forward. We might discover some of them kind of early on, but we don’t have all the other prerequisite technology that like makes it work. You know, with neural nets, some people thought that that was not going anywhere. It was a dead end and they don’t really work. I

02:11 believe it was actually Geoffrey Hinton who kind of went against the grain and a lot of the other expert opinion field in the ’80s and ’90s saying, “No, this is the way.” The reason they didn’t work is we just didn’t really have the compute for it. The hardware was lagging behind. And then here comes the Jensen Huang, builds the GPU, and now these things together are like, “Wow.” Don’t take that too seriously. I’m exaggerating wildly, but you get what I’m saying. We had this idea, but we didn’t have the hardware to run it, or at least run it in such a way that it produced amazing results. And of course, this all led to

02:41 LLMs eventually, large language models, as well as of course things like AlphaGo, AlphaFold, all of that stuff. And of course, with LLMs, some people, some scientists, yeah, researchers, they’re like, “It’s horrible. It’s not going anywhere. It will never achieve anything. It’s the wrong way to AGI. Shut it all down.” And some people are like, “No, you know what? It seems to be working. Let’s keep trying to increase it and improve it. Let’s see where this goes.” And of course, they worked pretty well, and then these people are very very mad, but they will not admit that they were wrong. What does that have to

03:13 do with DreamRSI, this Google paper that we’re covering? I apologize if that was a tangent, but I thought that this would help illustrate the point. The question is, how do we determine which experiments to pursue? Which ones do we contribute more energy and compute and resources to? Which ones are dead ends? How do we make better decisions about that? So, in this DreamRSI paper, the Google researchers, and it’s a collaboration with other universities, etc., so it’s not just Google, but they said, “You know, all those experiments and stuff that that we did, we kind of see it as just like ancient history, but

03:44 it’s also data. It can show us which experiments worked, which experiments were kind of dead ends.” So, if we create kind of this little branching chart, like this tech tree and run a bunch of AI agents through it, we can probably figure stuff out. The brilliance of this is that this really is almost free. It’s extremely cheap because we already have all of the data. We know what happened. Here’s how they kind of explain it. History is the world to dream in. And dream they use it as kind of simulation, right? So, we’re kind of simulating a certain history. We roll it back to a specific point and

04:17 then this AI kind of like goes through it. So, they’re saying that a learned world model and not approximation. The discovery tree the agent already built is an exact simulator of the search space it reached. A world that came free as a byproduct of working. So, going back to our little timeline here, historical tech tree. You can imagine going back to some point, let’s say in the 1900s. This thing is super laggy. So, I’m just going to leave it wherever it’s sitting. I can’t stop messing with it. The point is and actually Demis

04:48 Hassabis proposed this a sort of one of the thought experiments. If you took an AI model, you trained it up on all the data up until let’s say the early 1900s, could it then let’s say go through and maybe conduct some experiments or come up with the theory of relativity, right? So, if you train it up to somewhere here and you had it kind of go through foreign time, would it figure this out? Or a different question more closely related to this paper is like if we send it back in time and it went through history, could it like speedrun discovery? Could it find better ways to approach this tech tree? Could it find

05:20 faster avenues, better branches? Or another way to think about it is like could it get better at like could it develop the skill of improve the skill of figuring out how to go through these tech trees faster? So, that’s what they mean here, right? They took all their findings, their quote-unquote history and kind of like which findings came out of which one, which one was like a prerequisite. And so, dreaming is how it self-improves. So, a new exploration policy is tested by imagining it, not by living through it. Thousands of candidates dreamt against that world at zero executions and only the winner is

05:53 ever deployed. You might have seen those videos where like AI learns to play Mario or something, right? So you get a billion Marios and they all run through this thing and you sort of notice them kind of converge on a certain tactic that’s working, but instead of hitting your head on the questionable blocks and stomping whatever those things are called, instead of doing that, they’re trying to figure out how to best do RSI recursive self- improvement or or at least how to best do AI research. What do you go after first? Which sort of branches are, you know, not feasible or not worthwhile? And finally, every lap

06:25 adds a world. The wing policy goes back online and records a new discovery tree. One more world reaching places no earlier policy would have gone. The pool grows every lap. A policy dreamt against more worlds beats one tuned to the luck of a single run. And that policy brings back worlds no earlier one could. That is the recursion. So what decides whether those cycles are worth their computes is exploration. So where do we branch? What experiments do we try to run in parallel? When to cut a line off, right? It’s kind of like when we figured out what an LLM was and then some people were like, “But that’s the wrong way to

06:57 go. Just stop working on it. We’re we’ll never get there.” Like, what if we listened? And so what they’re saying is that exploration is one component still handwritten and frozen. So in other words, we can write some script that decides which avenue to pursue. Maybe it’s analyze the two different decisions in front of us, kind of grade them on some scale and then see which grade is higher and then just pursue that. But the point is a fixed strategy cannot learn from the experiences it accumulates. So it keeps paying for directions that have already failed. And optimizing online walks into two walls

07:28 at once. One is meta-level feedback is delayed and expensive. Meaning if you’re waiting for all these tests to be run, all the experiments to be completed. So first of all, it’s delayed because, you know, you have to wait till the end of it. Two, it’s expensive. You can’t let somebody take a stab at it cuz they might spend a lot of money and resources and computes, you know, going down these less optimal paths. And they’re saying it’s the only expensive because judging an exploration policy means watching it steer an entire discovery run to the end rather than scoring one candidate. And the meta policy space is vast. So, most

07:59 of the policies that you would have to try are bad ones. But, interestingly, and this came up for many different times in the various interviews that we did on this channel and just in general, something about AI being very connected somehow to simulations. Some people proposed the idea that maybe we are in a simulation and maybe there are millions or billions or an infinite number of other simulations being run. The joke I always make is like, what if some base reality out there wanted to figure out how to unleash superintelligence, but they wanted to do it in a very safe way? So, they spun up a thousand different

08:31 simulations and they recreated Earth just as it was approaching the time when AI would come online. They’re like, “Ah, let’s let’s start them in the ’90s or something like that.” And now they’re carefully observing us to see which one of us is going to figure it out, which one of us is going to figure out how to launch superintelligence safely because obviously that would be very valuable data for the sort of a base reality. The reason I bring that up is because us building simulations is almost a guarantee. There’s so much of it that’s kind of a pointed in that direction. I mean, what we’re seeing here is a very like rudimentary simulation of the research that they were already doing.

09:03 And so, they’re saying history is already a simulator. A simulator is anything that can answer what would happen if, you know, without running the world. For exploration policies, that object already exists, fully built and fully paid for, right? That’s just the history of the experiments that were already run. Think of an agent crossing unfamiliar terrain for the first time. It takes inefficient routes, hits dead ends, backtracks, and in doing so, it draws a map, whether a real map or they’re probably referring to kind of a neural mind, kind of an understanding of the world, where to go. The next policy does not have to walk the ground again. It reasons over the map. It avoids the

09:36 known dead ends, reconsiders decisions to first policy committed to too early, and compares whole routes before moving. And so, long horizon discovery has exactly the structure, and the map is already being written. So, if you think about it again, that’s why I showed that kind of like humanities tech tree, so to speak, so you can think of that as sort of how we discovered things, how we got up to, you know, today. And so, obviously, we’re we’re also continuing that. So, over the long horizon, we’ll draw more and more of this. But, if you think of it as sort of unfamiliar territory, then certainly we can think

10:07 of us understanding how the tech tree sort of grows and branches as drawing a map of it. And of course, us kind of running through the simulation, it returns immediately and costs zero executions. And as we have new experiments that get added to this kind of vast space of a potential branches, so the agent that that’s running this owns not one world, but a growing pool of them. So, the worlds evolve along with the agent. So, at the end of day, what happened? Did this approach work? So, here RSI is compared against other

10:37 machine learning models. So, for example, these two are machine learning libraries for building these predictive models, and basically lower is better. We want the lowest score possible because that represents how much resources we put into it. It’s better able to find the most optimized path, right? It’s like if we threw it some back into the prehistoric times and had it run the simulation of the tech tree of the actual world. Let’s say I would come back and say, “Well, what took you guys whatever, 20,000 years? I did it in under a thousand.” So, the point here is this Dream RSI is getting some of the

11:10 best scores. Gemini RSI with Gemini 3.7 Flash has the best score out of all the other models it was compared to, models or approaches. What’s interesting here is that this approach, it it’s adaptive. So, when everything is going great, it doesn’t use as much compute, but as progress slows down and things get a little more difficult, its compute, its effort increases trying to guide the agent forward or giving it some advice from its past, actually makes things worse. That sort of open-ended exploration works best. So, the reason

11:41 that it’s important is because things like Alpha Evolve, that was out of Google, we covered it a year or two ago, that was the big idea behind that. More of we took these at large language models and we had them try to optimize some branching evolutionary path. And they found that there’s certain uses where these approaches with large language models are ex- tremely extremely useful. Google has this massive global pool of resources that it uses of compute and Alpha Evolve, one of the things that it did, it found a better, more efficient way to sort of organize that. Specifically, this was improving the data center scheduling or or Borg, as they call it. We’ve covered

12:15 this, but I’m forgetting a lot of it cuz it was quite some time ago. But, it was extremely impressive in that it wasn’t just theoretical. Google was able to save what I assume it is millions and millions of dollars because of this optimization that this AI model came up with. You can think of this as Google basically producing more and more tools to do this sort of research. Alpha Evolve is one. And by the way, notice it beats Dream RSI on some of the mathematical optimizations that it’s doing. But, you can think of them as different tools working on different aspects or different projects. And I

12:47 think where this is going is that the goal is to build out one person that’s doing or one machine, one system that’s doing the experiments, one that’s figuring out how to best schedule those experiments, which one runs next. And over time, have these AI systems kind of predict where to go, what to do, then executing those experiments, and all the systems in combination improving the next generation of models. And of course, if those next generation of models are now better able to improve the tools that they’re using, as well as the processes, they’re better able to optimize all the systems involved, well,

13:21 that’s RSI, recursive self-improvement. So, definitely a very exciting time to be alive. Congratulations to Google. This does it seem like something that’s kind of foundational. It’s not as quite immediately exciting as a new model rolling off the shelves that are just amazing. This is more at the base at the foundational level. These are the things that’s going to allow us to continue improving the models moving forward. And of course Google is instrumental for a lot of this stuff that we’re seeing with AI progress. They invented the transformer and it is sometimes hard to tell which papers will translate to

13:53 massive impact in the future. This feels pretty big. Certainly the concept of taking the history that we already have. We have that data. We we know what happened in the past and then creating these assimilations, these worlds and having these agents run through them to figure out if there are more optimized paths that we could have taken. That feels like it’s a pretty big direction. Again, this is one more step towards us building, I believe, full-blown assimilations. Let them run because the data of course would be invaluable. So, congratulations to Google. Let me know what you thought about this. If you made

14:25 this far, thank you so much for watching. My name is Wes Roth and I will see you in the next one.