He Finally 10x Claude Code With This Method

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: He Finally 10x Claude Code With This Method · Channel: AI LABS · 12:21 · Watch on YouTube

Topics: AI agents, Task and workflow automation, AI evaluation and benchmarks, Claude Code, Software engineering with AI

Summary

The video explains Andrej Karpathy’s agent loop approach and how AI Labs adapted it to build app features with Claude Code. Their workflow uses project-context and build skills, pre-approved checks that the agent cannot edit, and a fresh feature-builder agent for each feature. In a restaurant-ordering example, the checks passed but missed the need for a customer-facing form, revealing that an agent can satisfy a score without completing the whole feature. To address recurring mistakes across features, they added an “auto loop” that reviews results and updates the workflow instructions while leaving the checks locked.

Key points

Karpathy’s method lets an agent run repeated experiments toward a goal, keeping changes when they improve a score and undoing them when they do not.
Loops are most useful for tasks that recur, have adequate token limits, can be evaluated with a clear score, and can be run by the agent so it can observe failures.
AI Labs uses loops for one feature at a time or for simple app versions, rather than asking a loop to build an entire app.
A project-context skill stores app details and conventions, loading its full contents only when needed.
The build skill uses a write-checks skill to define tests before implementation; approved checks are locked so the agent cannot weaken them.
Each feature is assigned to a fresh feature-builder agent, and round results are recorded for later review.
In the food-ordering example, all 11 checks passed, but the workflow also needed to ensure the website had an order form connected to the ordering rules.
The auto loop reviews prior results, identifies recurring mistakes, and updates the workflow instructions in program.md for future features.
In the project-management app example, the auto loop added habits to connect features to the app and to check older parts of the app for conflicts with new behavior.
Checks caught a broken mentions feature and an incorrect division of shared-project totals; the builder corrected these in later rounds.

Chapters

Start	Chapter	Summary
00:00	Karpathy’s loop and how it works	The video introduces agent loops and explains how Karpathy’s Auto Researcher repeatedly tested training changes against a score.
02:33	When to use a loop	The speakers outline four conditions for using loops and explain why they use them on measurable tasks, one feature at a time.
04:06	Building the feature workflow	AI Labs describes its project-context, build, and write-checks skills, along with locked checks and separate feature-builder agents.
06:39	Running the first loop and finding a gap	After a sponsor segment, the team tests its workflow on a restaurant pickup-order feature and finds the checks missed the customer-facing form.
08:46	Adding an auto loop to improve future runs	The team explains how an auto-loop skill reviews recorded results and updates program.md while keeping the checks protected.
10:49	Testing learned habits on shared features	A project-management app test shows the auto loop adding guidance to prevent recurring integration and compatibility mistakes, with checks catching further errors.

Quotes

In that time, the agent ran 700 experiments and found 20 changes that made the model train faster. — 01:32

We never let a full app get built in one loop and call it done. — 03:35

This makes sure that the checks are written before any feature is built so that the agent has something to verify against instead of judging its own code. — 05:06

The fix for that is a second loop that reads how the first loop ran and changes how the first loop works. — 08:46

Whenever you’re working on the next feature, you don’t have to face the same issue again. — 12:21

Concepts and entities

Karpathy loop, Auto Researcher, Andrej Karpathy, Claude Code, project context, write checks, feature builder agent, auto loop, program.md, Upstash Box, Shopify, AI Labs

Full transcript (original language)

00:00 People have been using loops to get AI agents to build a lot of things on their own. And the reason that works is the structured workflow behind each loop. That’s why people let their agents work on big tasks for hours at a time, and they’re getting good results. There are a lot of ways to set one of these workflows up, and one of them is Karpathy loop. So, we set up the Karpathy loop on our own apps to see how well it actually holds up. It held up well, but we found one issue that the loop can’t see on its own. To fix that issue, we made one change to the loop. If this is your first time, we’re

00:30 software company, and this is our channel AI Labs. And in this video, we’re going to show you what the Karpathy loop is, how to set it up on your own apps, and the change that fixes that issue. But before [snorts] we go into the workflow, let’s go over Karpathy’s method itself, because this loop is built on it. If you don’t know loops, we already have a full video on them on our channel. You can watch it so that you understand loops better. Now, Andre Karpathy made a project called Auto Researcher a while back, where an AI agent kept working on training a model and ran the code again and again until the model actually improved. The

01:00 agent did this in a loop, so there was no need to direct its steps. Now, in Auto Researcher, the agent was only allowed to change one file, and that was the file that trained the model. Every round was one experiment where the agent made one change to that training file, and then trained the model for a few minutes. After that, a second file gave the model a score. If the score got better, the agent kept the change. But if the score stayed the same or got worse, the agent undid the change. The agent wasn’t allowed to touch the scoring file. This was because if the agent could change the scoring file, it could have just made the score easier to

01:32 beat instead of making the model better. And a third file called program.md told the agent how to run each round. And program.md was basically an instructions file that Karpathy wrote in plain English. Karpathy wrote that instructions file and let the loop run for 2 days. In that time, the agent ran 700 experiments and found 20 changes that made the model train faster. Now, this was an experiment that showed how agents can keep going and check every single possibility, which wouldn’t be possible if you controlled every step yourself. This is because controlling

02:02 700 experiments yourself takes a long time, but the agent ran them on its own and found different methods that worked. The CEO of Shopify also ran the same kind of loop and got good results. He ran it on one of his own models while he slept, and the agent ran 37 experiments. In the morning, his model was performing 19% better. So, Karpathy’s method showed that when an agent is just given a goal, it can keep trying things and get better on its own without you directing every step. This is when a lot of people started getting into loop engineering. Now, before [snorts] you set up a loop,

02:33 you need to know when you should even use one in the first place. If you use loops for the wrong kind of task, you waste tokens and might not even get what you want. Loops are really helpful when they’re placed in the right scenarios, but the right scenario depends on what you’re actually trying to get done with the task. For a loop to be worth setting up, the task needs four things. The first is that you repeat the task often. A loop takes time to build, and you only get that time back when you keep repeating the task. So, for a job you do once, one good prompt is enough. If you set up a loop for that kind of job,

03:04 you’ll spend more time on the setup than on the task itself. The second is your usage limit, because loops use a lot of tokens. In a loop, the agent reads your project again and tries a new fix on every round. That uses tokens even on the rounds that fail. So, a long loop on a $20 plan hits the usage limit before the work is even done. The third is that the work can be checked without by something that gives a clear score. In an app, that’s done by small pieces of code that try a feature and confirm it works. The last is that the agent can actually run what it built and see what

03:35 breaks, so it knows what to fix in the next round. We use different types of loops in our workflows, but only for tasks that have a clear score we can measure, and we never let a full app get built in one loop and call it done. We only use loops to build one feature at a time or a simple version of the app that the loops checks can verify. But before we set things up, it would be great if you subscribe to the channel and hit the hype button. This small gesture of support goes a long way for us. Karpathy [snorts] has shared a lot about how he works with AI agents. So we took what he

04:06 shared and turned it into a setup we could use in our own workflow. In order to make his loop work for our own app, we set up a few skills and agents so that we could automate this job. We’ll explain what each part does in depth so that you can ask Claude to build each one for you as well. The first thing we built was a skill that’s really important called project context. This skill is specific to whatever you’re building and contains everything there is to know about it. It acts as a memory bank for what the app does, which pages it has, the conventions it follows, and the things it should avoid. So when the

04:36 model is working, it can pull context from there. The project context skill grows as the app grows. So it’s always up to date with the latest version. We use a skill instead of a files where this information would normally live because only the skills short description stays in window all the time. That means the agent always knows it’s available, but only loads the full details when needed instead of carrying all of them into every task. So whatever you’re building, you can ask the agent you’re working with to set up a skill that stores all of this context for you.

05:06 After that, there’s a main skill that drives the whole thing so that the loop can run without you watching it. It’s the build skill and it has all the instructions for running the loop on each feature. The first job of the build skill is to write the checks and it doesn’t do that on its own. It uses another skill called write checks. Write checks is similar to the test author agent we used in our previous videos, but it’s tailored to this project and workflow. This makes sure that the checks are written before any feature is built so that the agent has something to verify against instead of judging its own code. Once the checks are written,

05:37 the build skill lists what every check tests in simple words. This is where you need to go back and forth with your agent so that you can make sure the checks cover all the problems that commonly happen during the work. Once you give it a go-ahead, a small program called approve checks locks them in. The approved checks program moves the checks into a locked folder, and a rule in Claude code settings blocks the agent from editing that folder. So, the loop can’t make its own checks easier. Once you approve the checks, the build skill commits them. This adds another layer of protection in case the agent tries to

06:08 change them. Then, it runs the loop, but the build skill doesn’t build the features in its own context window. To keep each feature’s context separate, it gives every feature to the feature builder agent. The feature builder agent starts fresh each time and builds the feature as instructed. The result of every round is stored in a file so that we can use it later on. When every feature is done, the build skill gives out a report, and all the loop’s instructions like the fixed rules and how to work are written in program.md. This follows the same format Karpathy used, too. That makes our setup complete

06:39 for running the loop this way. But, before we run this setup, let’s have a word by our sponsor, Upstash Box. Agents can build and fix real software now, but your laptop can only run one at a time. Because running more than one means they’ll either collide with each other, slow everything down, or worse, they’ll quietly read or change files on your machine. That’s why you’d want a sandbox, and it’s exactly the problem Upstash Box fixes by giving each agent its own sandbox. Each one is a real computer in the cloud with its own shell and file system. So, an agent can work on your repo without ever touching your

07:11 machine. We spun up 10 of them, each with its own job. One built a new feature, another chased down a bug, and none of it touched our laptop. Because each Upstash sandbox saves its own state, they kept working for hours, even overnight. It’s a whole software factory, and none of it lives on your machine. There’s a free tier, you only pay for what you use, and it scales to zero when nothing’s running. Have multiple agents powering your workflow at upstash.com, links in the description. Now, since we were working on a project that was already half-built, we prompted Claude to use the build skill and asked it to build a

07:43 feature that lets guests order food for pick up from the restaurant’s website. But that feature wasn’t in our feature list yet. So Claude added it and wrote the rules for online ordering into features.md. This was the file that we were using for feature tracking. With the rules written down, Claude gave us all the checks it was going to make. The first list had 10 checks. We wanted the guest’s email as well. So we asked Claude to add it. That made 11 checks and the new check made sure the email was a real address. When we approved the checks, Claude committed them. Then the feature builder agent

08:15 built the feature and it took about 6 minutes. At the end, the report showed that all 11 checks passed in the first round, so the loop didn’t need another round. But the loop also caught something the score didn’t show. The checks only tested the ordering rules. So the loop had written those rules, but not the order form that guests use on the website. So the loop built the order form and connected it to those rules before it marked the feature as done. But this kind of loop has one problem, which is that the agent has habits. In a loop, if it sees that a particular way wasn’t working, it’s going to improve it

08:46 in that loop, but it won’t remember that for the next time it runs that loop. Our loop has the same problem because every feature starts with a fresh feature builder agent and the same files for instructions. So a mistake that a model made in one round is going to carry over to the next. The fix for that is a second loop that reads how the first loop ran and changes how the first loop works. We built our own version of that second loop as a skill called auto loop. In Carpen these method, a person writes program.md, but in our setup, the auto loop rewrites the how to work part of program.md for us. To do that, the auto

09:18 loop reads results file, which stores every round the loop ran. For every round, results file also shows whether the round was kept or undone, what the agent tried, and which checks still failed. The auto loop basically checks for problems that keep coming back like the same mistake agent was making or the same gap showing up in two features. Then it writes down each habit with the rounds that show it. Then the auto loop rewrites how to work part in the program.md, so the next features are built with the new instructions. But the auto loop can’t edit the checks. If the

09:48 auto loop could edit the checks, it could make them easier to pass. Then the rounds would stop failing without the habits ever getting fixed. Now, to test the loop within the loop, we ran one on another project. It was a project management app that didn’t have any way of people combining and sharing their workflows. To run it, we asked Claude to create another skill called auto loop. The auto loop skill basically runs the same loop with the same feature builder agent and write check skill that the previous setup was using, but it runs that loop inside another loop. This is different from the build skill that we

10:19 used previously because the build skill only runs one pass. Every feature gets the same instructions and the agents build it, but nothing the loop learns carries over to the next feature. The auto loop skill also runs the loop one feature at a time, but after each feature, it reviews and updates the instructions, so the next feature starts from the method that actually worked. This helps make your loop much better. The whole purpose of this skill is actually to improve the loop itself and make changes in the instructions. So, whenever the loop runs the next time, it

10:49 works better than the previous run. So, to add features to our app, we ran the auto loop skill and told it what we wanted. We asked for a way for many people to work together, so they could share focus sessions and run projects together. The auto loop skill then started by adding those features to the tracking file. Just like before, the auto loop skill first lists the checks. You can change them if needed or ask your agent to start building directly. The first feature was a shared database, so everyone’s work could live in one place. The database passed all 10 of its

11:19 checks, but the app never saved anything to it. So, the auto loop added a habit to the instructions, which was to connect each feature to the app in the same round that makes its checks pass. Then the auto loop spotted a second problem with mentions, which let you assign a task by typing someone’s name after an at sign. Mentions passed its checks, but an older part of the app still read the at sign the old way. That’s why the auto loop added a second habit, which was to find every place the app already does a job and make each one follow the new rules. Both habits worked

11:50 because the next features came with their screens already connected. To see the checks in action, the builder got two rounds of shared projects wrong. In the first round, it broke mentions, so the mentions failed and the round was undone. In the second round, it estimated each person’s share the simple way, so three people got 33 each, which only adds up to 99. That check failed, and in the third round, the builder fixed it and passed all 10 checks. When it is done, it gives a report telling you where the agent went wrong and how it fixed that, along with what the workflow looked like. So, whenever

12:21 you’re working on the next feature, you don’t have to face the same issue again. Now, all the skills that we created in this video, along with other resources, are all available in our community, AI Labs Pro. So, if you found value in what we do and want to support the channel, this is the best way to do it. The link’s in the description. That brings us to the end of this video. If you’d like to support the channel and help us keep making videos like this, you can do so by using the Super Thanks button below. As always, thank you for watching and I’ll see you in the next one.