System 1 vs System 2 Explained

Author: Prompt Engineering · Watch on YouTube

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: System 1 vs System 2 Explained · Channel: Prompt Engineering · 10:10 · Watch on YouTube

Topics: AI agents, AI evaluation and benchmarks, Prompt and context engineering, AI models

Summary

The video explains System 1 as a model’s single forward pass and System 2 as the same model working through multiple passes while writing and reusing intermediate notes. System 1 is effective when an answer is apparent in one look, but can substitute an easier question when a task secretly requires several steps. System 2 can solve such tasks by building on results from earlier steps, though giving it more time does not make it inherently smarter. The video recommends choosing based on how many steps a problem requires, and describes routing low-confidence System 1 answers to a thinking model, with a tuned confidence threshold.

Key points

System 1 answers in a single forward pass; System 2 runs the same stack of layers in a loop.
In System 2, the model writes intermediate steps and reads them back on later passes.
The manager-chain example shows how a one-look model can follow the right chain but stop short of the correct answer.
System 1 is well suited to questions whose answers are visible in one look, such as classifying a ticket.
When a question requires multiple steps, System 1 may substitute an easier question for the one asked.
Breaking a complex question into smaller one-look questions and combining the results in code can improve performance.
On the chain tests, System 1 confidence fell along with accuracy as the required reasoning depth increased.
A combined system can use System 1 for initial answers and send low-confidence cases to a thinking model.
Confidence scores should not be blindly trusted; the threshold should be tuned for the application.
Choose between the approaches by counting the steps needed to reach an answer.

Chapters

Start	Chapter	Summary
00:00	The manager-chain example	A manager-chain question illustrates how a fast, one-look model can stop short while a thinking model reaches the correct answer.
02:08	One look versus a loop with notes	The video defines System 1 as a single pass and System 2 as repeated passes that build on written intermediate results.
04:46	Testing longer chains	A chain-length experiment shows one-look models slipping as tasks require more steps, while the thinking model stays near perfect.
06:26	When each system is useful	System 1 is useful for answers visible immediately but can substitute an easier question when a task hides multiple steps.
08:35	Combining systems and choosing a model	The video recommends routing low-confidence cases to a thinking model, tuning the threshold, and selecting by the number of required steps.

Quotes

System one is the part of you that answers on autopilot. — 01:37

System one is one look, system two is the same look in a loop with notes. — 04:46

This does not make system one weaker. — 06:26

If the answer is visible in one look, that’s your system one. — 09:40

Concepts and entities

System 1, System 2, Thinking, Fast and Slow, Daniel Kahneman, chain of thought, confidence score, Transformer

Full transcript (original language)

00:00 Peter Liljeqvist, everybody’s talking about system one and system two models. But I see a lot of confusion about what the difference between them actually is. And when you would use which one. So, in this video, I really want to clear that up. And we’re going to not only talk about system one and system two models, but when to use which. Now, to make it concrete, let’s start with a very simple example. Here is a list of who manages who at a small company. There are three separate chains of

00:32 people, and the facts are shuffled around intentionally. So, you can’t just read down the list. Now, the question is very simple. Start at Mo and go four managers up. Who do you end at? I gave this to two models. The first is Chev, which gives you an answer without writing anything out, almost instantly. And it says Eli. The second is current 3.8 with thinking turned on. It basically wrote its chain of thought,

01:05 and the final answer that it came up with is Gus, which is correct. But the thing is that Chev was not guessing. Eli is exactly three levels up from Mo. Now, Chev was on the right track, but it stopped one step short. And this missing step is the real difference between system one and system two thinking. By the end of this video, you will know why it stops there. And what that means for your own problems if you’re selecting which type of models to use.

01:37 Now, these terms come from Daniel’s Kahneman’s book, Thinking Fast and Slow. System one is the part of you that answers on autopilot. You see a face, you just know it’s angry. System two is the part that works things out step-by-step, like doing mathematics, where you have to hold on to the results along the way. In a model, system one is single forward pass.

02:08 Your input goes through a fixed stack of layers only once. A good mental model of this is to think of it as one look at a problem. Whatever the model can work out in that one look, it has to answer that. Now, there are problems which will need more than one look, and this is where system one models fail. Geve is basically built around this whole idea. Your text and your typed questions go

02:38 in. Then the model runs a single pass, and you get back a probability for the answers you allowed. When they say it cannot hallucinate, it has to pick from the answers that you have provided. So, it can be confidently wrong, but it’s going to answer only from the selected options. Now, theoretically, a chat model with thinking turned off works the same way. Every token is one look at the problem,

03:11 and it doesn’t have any room to work anything out first. So, in our example, when I asked Gwen, it answered Tia, which is just one level up. So, what happens when a question needs more than just one look? That’s where system two comes into play. In a model, system two is not a different kind of brain or a different kind of architecture. It’s the same stack of layers run in a loop. Now, the difference is that each time

03:41 around, the model writes one step down, and on the next pass, it reads that step back. This is basically what the chain of thought looks like. Now, the whole idea is that at every step the model takes notes and on the next step the model can reuse those notes. So, in our example, in the first pass the model might write something like Mo’s manager is Tia. And then in the next pass it’s going to read Tia and find Tia’s manager.

04:14 So, every step here creates a result that did not exist before. And the next step basically builds on top of that results. Just like when you’re doing mathematics, for example, multiplying 17 by 24. There’s a very interesting paper on this showing that checking whether two points in a graph are connected is probably out of reach for a transformer that has to answer straight away. And our manager chain is exactly that

04:46 kind of problem. Now, the interesting thing is that if you let the model write out its step, it can answer those type of questions. Okay, so here’s a very simple way to think. System one is one look, system two is the same look in a loop with notes. So, here’s a fun experiment that you can run yourself. I give Jeff, friend with thinking off, and Fran, friend with thinking on, the same kind of question

05:16 with chains getting longer and longer. With a single step, they all get it right. That’s just a simple lookup. But as soon as the second step is needed, both one look model starts to slip. And a few steps in, they’re getting almost nothing right. The thinking model stays near perfect the whole way. The only thing that grow is its notes. But the thing is that the model is not getting smarter if you let it think for longer.

05:51 It’s just taking more steps with nodes. Now, when I was looking at the results, I found something really interesting. So, remember Eli, when I looked at where Jev’s wrong answers landed, most of them were not random. Whether I asked for two, three, or four levels, its most common answer was the person three levels up. It walks up to the right chain and runs out at about the same place almost every single time. My guess is that’s what one look

06:26 looks like from the outside. It gets you about this far and no further. We do not know what’s inside Jev, so I can’t really say why limit sits there exactly. But, one very important takeaway, this does not make system one weaker. It’s the right tool whenever the answer is visible in one look. Things like, is this ticket about billing? Is it urgent? Does it mention

06:56 any fund? You can ask a one-look model lots of questions about that simple input all at once. And this is where Jev is actually really good. Where it slips is when you hand it a question that secretly needs steps. Daniel calls this substitution. System one quietly answers an easier question instead. When I asked Jev, is this email fishing? on a hard data set that I mentioned in my previous video,

07:31 it was as bad as a coin flip because it was answering, does this looks like fishing? But, when I broke down the same question into smaller question where the model was looking at a specific feature and then combine the answer and code, the accuracy jumped significantly. There’s one more thing Jeff gives you with every answer, which is the confidence score. This is where it becomes even more

08:02 useful. So, on the chain questions as they got too deep for one look its confidence dropped right along with its accuracy. This is exactly how Daniel describes these two systems working together. System one answers first, then system two steps in when something feels off. Now, the mental model to think about this is that you can design a system where both system one and system two levels are going to be working together.

08:35 Now, in this setup you use one look model to answer everything and you keep the confidence score. Things that are lower than a certain confidence threshold, they are going to be handed over to the thinking model. This can be things like tool calls in a inter-tick harness where system one makes the tool calls but on tool calls where it’s not confident, you pass it back on to system two or thinking models which can actually look at it more comprehensively and then make a

09:07 decision. And this is how I think most of the future systems are going to be designed. However, you can’t blindly trust the confidence score that it is outputting. It’s skilled probability, so not really the confidence that the model is assigning to it. So, for any application, I highly recommend to fine-tune that threshold before you decide to use it. Okay, so how do you select between system one and system two models? Now, the easiest mental model is just

09:40 count the number of steps that the models would need in order to get to an answer. If the answer is visible in one look, that’s your system one. If you need a result you don’t have yet, and the model has to create the results in multiple different steps, that’s where you want to use system two. Or you break the question into one-look pieces yourself and then combine them in code. In the next video, I’m going to build

10:10 this into an agent where the thinking models does the planning and Jeff handles the hundreds of small decisions in between. The code for the chain tests is in the video description. So, if you want, you can play around with it and see how further you can push a system one model. Anyways, I hope this was helpful. Thanks for watching and as always, see you in the next one.