How to Build Agents That Improve Over Time: Model, Harness, Context | Interrupt London

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: How to Build Agents That Improve Over Time: Model, Harness, Context | Interrupt London · Channel: LangChain · 20:09 · Watch on YouTube

Topics: AI agents, AI evaluation and benchmarks, Prompt and context engineering, Software engineering with AI, AI models

Summary

The talk presents a framework for building agents that improve over time, defining an agent as a model, a harness, and context. Each component can be updated through human-led, human-approved, or eventually automated processes. Traces and evaluations form the foundation of the improvement loop: traces show what happened, while evals define what good looks like and help detect regressions. The long-term vision is for agents to identify their own mistakes, create evaluation data, and propose improvements to their models, harnesses, or context.

Key points

An agent consists of a model, a harness that manages the agent loop, and context such as files, knowledge, and tools.
Agents can improve by updating any of these three components, with processes ranging from manual changes to full automation.
Supervised fine-tuning uses selected successful traces and is suited to simpler, narrowly defined tasks.
Reinforcement learning trains through task rollouts and scoring, offering a higher ceiling for complex agent tasks but with greater complexity.
Harness improvements involve analyzing traces and changing code, prompts, or tools; LangSmith Engine proposes fixes while keeping a human in the loop.
Context is comparatively easy to update and can evolve at the core-agent, organization, or user level.
Traces capture exact inputs, outputs, and event sequences; the talk distinguishes runs, traces, threads, and trajectories.
Evals define desired behavior, including expected answers, criteria, tool-call sequences, and the resulting state of an environment.
Realistic evals may require simulated environments, and red teaming can proactively probe agents for issues such as correctness and security.
The envisioned improvement loop uses feedback and traces to create eval data points, then proposes changes while checking for regressions.

Chapters

Start	Chapter	Summary
00:00	What an agent is and how it can improve	The speaker defines agents as a model, a harness, and context, and outlines manual, human-approved, and fully automated improvement.
03:07	Improving models and harnesses	The talk compares supervised fine-tuning with reinforcement learning, then describes trace-based harness updates and LangSmith Engine.
09:17	Traces as the basis for improvement	The speaker explains trace terminology and why agent observability needs tools designed for semantic and conversational data.
12:23	Evals, criteria, and simulation environments	Evals define good behavior through realistic inputs and criteria, while simulated environments enable safer end-to-end testing.
16:01	Finding mistakes with red teaming	The talk covers ways to discover agent failures and describes proactive probing, including checks for correctness and other risks.
18:38	The agent improvement flywheel	The speaker summarizes a feedback loop that turns traces into evals and proposed updates, with self-improving agents as a future goal.

Quotes

We think there are three parts to building an agent. — 00:32

We think these are the core foundation of this improvement loop, and it’s where we spend a lot of time. — 09:17

So evals are where you define what good looks like. — 12:23

You then want to add that as an eval data point so that as you try to improve, you can make sure that you’re actually improving on that and not regressing against previous ones. — 19:09

Concepts and entities

LangChain, Deep Agents, Claude Code, Codex, LangSmith Fine-Tuning, SmithTune, LangSmith Engine, OpenWiki, SmithDB, Hex, Profound, Fireworks, Baseten, red teaming, traces, evals, fine-tuning supervisionado, RL

Full transcript (original language)

00:00 Cool. For this session, I want to talk about something that we think a lot about at LangChain, and that is definitely part of the future of where we think the industry and where we are going, which is building agents that improve over time. And so this is a North Star that, while doing this in a fully automated way, may not be anytime soon. This is a North Star that guides a lot of how we think about things at LangChain.

00:32 And so as I talk about this, I want to set the stage with talking about just like what is an agent and how do we think about improving it? Because you can’t improve it if you don’t define what it is. And so we think there are three parts to building an agent. First is a model. And so this is choosing the powerhouse of the agent. And there’s a bunch of different models that you can choose from. And some are good at different things. Some have long context windows. Some are really fast. But choosing one to be the core workhorse of an agent. And then the other thing I’d say is there’s other models at the periphery.

01:04 There’s other models for guardrails and evaluations and things like that that might be part of the harness, but not the core workload. But anyways, at the core of an agent is a model. Then there is the harness. So you could use Deep Agents, Claude Code, Codex. This is the harness of the model. What is a harness? This is the system around the model that makes it really useful. So the model does all the reasoning and decision making. But the harness is responsible for deciding what the model sees, for actually taking the actions of what the model decides to do,

01:35 for observing and verifying results, for persisting things onto disk or into databases, and is really related to all the control around what happens in this loop. We’ve built Deep Agents as an example harness for what this can look like. It encodes a bunch of best practices for what these harnesses should do. So it has an execution environment, which is where it runs. So this includes a file system and sandbox. It does a bunch of off-the-shelf context management. So it deals with skills, memory, summarization.

02:06 And then it has human in the loop as well as subagents for different ways for interacting with humans and other agents, respectively. And then the third part of an agent is context. And so this is markdown. This can be files. This can be knowledge that’s broad. It can be tools that it uses. But when I say context here, I mostly refer to kind of like text and other sources of knowledge that is brought. And these three things, we think, make up what an agent is. It’s got the model, you wrap it in a harness,

02:37 and you connect it to context. And so when we talk about agents improving over time, they can improve across any of those dimensions. So you can improve the model, you can improve the harness, or you can improve the context. And they can improve in a bunch of different ways as well. So they can improve manually. So they can be human driven. So you can have a human AI engineer go in and kind of like drive the cycle. And that’s a lot of what AI engineers do today. They can be human in the loop. You can have some background process that is improving

03:07 this agent over time. But you have a human approving things. Or they can be fully automated. And again, this is maybe the North Star or the Holy Grail. And I don’t think we’re there right now by any stretch of the imagination. But this is where we want to work towards. And so I want to talk about these three different dimensions. I want to talk about improving the model first. So models are generally improved by updating the weights of the model. Agents use different models, so you want to choose which ones to update. And so maybe you update the weights of the core model that’s driving the agent,

03:37 or maybe you update the weights of a guardrail or an eval check or something like that. This is almost always human-initiated. So the way that you update the models, and I’ll touch on that in a second, but the way that you update the models is generally pretty time and capital intensive. And so you almost always want to have a human going in and preparing and deciding to do this. It would be pretty costly to just do this continuously in the background or on a whim. And there’s two different methods that we see for doing this. One is called supervised fine-tuning, and the other is RL, or reinforcement learning.

04:08 Talking very briefly about supervised fine-tuning, at a high level, you take a bunch of traces. You filter to ones that you want the agent to learn on. And so these are generally ones where it does well. And then you basically train the model on that. And this is a really good option for simpler tasks. So we see people doing this a lot for classification or extraction or summarization tasks, where it’s pretty narrowly defined. And so one of the things that we launched recently is LangSmith Fine-Tuning

04:38 in the form of smithtune. smithtune is a CLI. It interacts with LangSmith. It pulls down trajectories. It filters them. It processes them into the right data format. We then hand off to Fireworks or Baseten or another one of these model providers that have, they actually have a bunch of off-the-shelf supervised fine-tuning, basically, APIs where you can just pass in data and get back a model. And so we plug into that and then get back an optimized model. And so we see people doing this, again, for kind of like simpler tasks.

05:09 For more complex tasks, specifically like that core model that’s driving the agent, we see a few people reaching for RL. So RL is a lot more complicated than supervised fine-tuning. There’s no managed offering for RL that’s really that great. So there are a bunch of managed offerings for SFT. There’s none for RL. You can generally do more things with RL. And the way that RL works is a little bit different. Rather than just giving it trajectories, you give it a bunch of tasks. And you do rollouts over those tasks. You basically run the agent over it.

05:40 You then score it in real time, backprop it, update the agent or the model that’s part of the agent, and then do more rollouts. And so a little bit different than SFT, a lot more complicated, but higher ceiling. The next thing you can update is the harness. The harness, how do you update it? Well, it itself is basically code and logic there. And so you’d update the code, or you’d update the prompts, or you’d update the tools. The general process for this at a high level is pretty simple. You look at a bunch of traces. You identify issues. You then make changes based on those traces.

06:13 You’ll probably want a human in the loop for this, because this will be changing core code of the agent, or core instructions of the agent that will usually propagate to all users. And so you probably want some human in the loop to review these things. But you can start to automate it. And so LangSmith Engine, and Ben talked about LangSmith Engine earlier, LangSmith Engine is our attempt to do this. So this is an agent running in the background. It basically runs over traces, monitors them, clusters them into basically different issues,

06:44 and then tries to propose fixes, put up a PR, add tests, add eval data points, things like that. Engine itself is nothing more than a deep agent in a sandbox with LangSmith CLI and some specialized subagents, some specialized skills, and some specialized prompts. And so this general process we think is an example of starting to automate, but not going fully there. Still human in the loop, very important part, but starting to automate some of this improvement of the harness. And then lastly, talking about improving the context,

07:15 this is the easiest to update. This is just text. And so it’s very easy to update this. And as a result, you can update this at different levels. So rather than just updating this at purely the core agent level, you can start to update this at the user or org level. And you can start to have memory at those different dimensions that evolves over time. And so similar to how you’d update a harness, the process is basically look at a bunch of traces, identify issues, and then make changes. And again, you’re making changes to the text, so it’s a lot easier to do.

07:45 There’s a few companies that I think do this well. One I want to call out is Hex. And so Hex is a data analytics agent. Part of their value prop is this unified context layer. This unified context layer improves over time. That’s a core part of the value prop. How do they do it? They basically do something similar to Engine. They run an agent in the background. So the Review Agent analyzes interactions for you and finds context gaps and suggests fixes. And again, it’s specifically for the agent that they’re building. But they basically have this background process, looking at traces, identifying things,

08:15 and then proposing fixes. Profound is another example of a company that I think does this well. And they actually do this at the user level as well. So you can see that they constantly synthesize data into Brand Records. That’s maybe like a company-level thing. But then it also collects Personal Records informed by interactions with your AI Marketer. So as you’re interacting with the agent, they have a process in the background that will create this basically personal memory. So these are examples of two companies that I think are doing a really good job of this. We’re experimenting with things in this vein.

08:45 We don’t have anything in the platform, but we do have an open source project called OpenWiki. OpenWiki is basically this idea of creating this knowledge or this brain for an agent that can be updated. So this is… This is built for coding. It started… It’s populated by looking at your coding repo. But we actually have integrations with your coding agents, so we can look at Codex or Claude Code traces and see what it struggles with, and then bring it back into OpenWiki. So what’s common across all of these three things? We think there’s two things that are common, traces and evals.

09:17 And we think these are the core foundation of this improvement loop, and it’s where we spend a lot of time. This improvement loop may look different for different use cases and different verticals. The hardest part of the improvement loop is oftentimes choosing what to remember and what not to remember and how to remember. And a lot of that is in the prompting, and so that’s very domain-specific. But we think there are other ways that we can help, namely with traces and evals. And so this is where we’re putting a bunch of effort. So traces, what are traces? We think there’s a few different aspects of traces.

09:50 So there’s a few different terms that we use when talking about traces. First is run. This is like a single invocation of a model or a single invocation of a tool. It’s basically a single step. You then add up a bunch of those runs and you get a trace. And this is a sequence of those steps. You combine a bunch of those traces and you get a thread. And so the way to think about this is if I message an agent, it goes and does a bunch of work. There’s a bunch of runs happening in the background. That itself is one trace. When I start having a conversation with it, I create this thread. And then trajectory, as we talked about in the keynote, trajectories

10:21 are a view over this thread in a standardized format. The important thing is that for all of these, you have the exact inputs, the exact outputs, and the exact sequence of events. And this is really needed, whether you’re going to be fine-tuning the model or trying to debug it with a coding agent and then update the code. Being able to see exactly what goes in, exactly what comes out, and the exact sequence is really important. We think traces are a new type of data. That’s why we built SmithDB, which is a database of our own

10:51 that we built to deal with this new type of data. We think there’s new types of failures here. The failures here are semantic failures. They’re logic failures. There are a bunch of issues that pop up that don’t show up in traditional logs. They show up in the conversational context. And so how you deal with those and how you triage those needs to change. And likewise, how you interact with the data, how you query it, also needs to change. And all of that is why we built SmithDB, which is our database that we

11:22 purpose-built for agent observability. We think a bunch about how to structure this database to be the best database for ingesting, querying, storing, visualizing data. The example of this that we launched today is this AI filter, this AI Search, where you type in, again, a question that you want to filter on. This is a semantic question. And under the hood, we score each trajectory individually according to whether it meets that filter or not. And so this is an example of how we

11:53 think about building this database that is optimized for this new type of data. So you can see what’s going on. You can query what’s going on. But next comes evals. So evals are really important for doing this loop for a few reasons. First, they’re the new PRD. And so this is a saying that I really like, where you might have a product requirements document describing what the product should do. In AI, there’s so many edge cases that the best way to describe that oftentimes is in the form of evals.

12:23 So evals are where you define what good looks like. And so good can mean you respond in a certain way to certain queries. It can mean you don’t respond to certain queries. Defining the limits, defining what good looks like is a really important part of evals and a really important part of defining how you want to improve. And so this is kind of like the guardrails that are going to keep you in the safe zone as you iterate in this improvement loop. Building evals is really, really hard. This is one of the main things that we work with

12:53 a bunch of folks on. There’s not really a silver bullet for evals out there. Sure, we can provide infrastructure for running evals and visualizing eval results. But evals need to be domain-specific, both the datasets that they run on and the evaluators that are used. And so we still think building evals is really hard. There’s different parts of this. So first, you need to come up with realistic inputs. And so this can often be hard. You might not know what people will even ask your agent. Sometimes they’ll go completely off topic. Sometimes they’ll go orthogonally or close to it.

13:26 But you may not be able to predict all these cases up front. And so even being able to just enumerate what realistic inputs are is not as easy as it sounds. And this is why tracing is so important. You need to be able to define what a good answer looks like. There’s different ways that we’ve seen people do this, maybe calling out kind of like three off the top of my head. One, you can define what the expected output is, a certain number that it should mention, an exact kind of like phrase that it should use. This is kind of called an exact match. This gets complicated for more complex agents,

13:58 because it’s usually hard to do that. And so what we see people do is then defining criteria. Basically, this agent should have part of the answer that mentions this. This answer should not say this. It could be positive or negative criteria. These criteria could act on either the final answer itself or the environment that the agent’s running in. So oftentimes, these agents are interacting with the environment. They might be changing databases in certain ways. They might be manipulating files in certain ways. And so you don’t want to judge its final answer.

14:28 You want to judge how it interacts with its environment, or more specifically, what the end state of the environment is. And the third thing that the criteria can judge is actually the exact sequence of tool calls as well. And so depending on your use case, you may just care that it ends with the right database state. Or you might care about the sequence of tool calls. And so you can write criteria for either of those. But defining what good looks like, this is oftentimes one of the hardest things to do and one of the things that’s best done by domain experts or subject matter experts. And then the last thing that I want to call out

14:59 is the simulation environment. So when you run evals, you need an environment to run them in. If your agent naturally interacts with production systems, you probably don’t want it hitting production systems as you run a bunch of evals, especially if those are doing write operations or actually changing the database. And so one of the things that we see people doing in order to be able to run these evals is coming up with these simulation environments where they can basically interact freely with either like a mock database or a dev environment that they spin up,

15:31 basically run these end-to-end evals that actually do interact again with this simulated environment and run that way. This can be tricky. Your agent may interact with a bunch of different services. You may or may not have kind of like dev or staging things for those. If the agent’s interacting with a system that you create, that might be easier. If it’s interacting with Salesforce, what do you do there? That’s not a system that you kind of easily own. And so we also spend a lot of time thinking about what these environments should look like.

16:01 And then on top of that, we also spend a lot of time thinking about how do you find agent mistakes. So again, there’s different ways to do this. You can look through how people actually use your agent and look for places where it fails. You can have domain experts think about how they would use the agent, try it out, and then see when it messes up. The domain expert one is kind of manual and costly. The looking at production traces one, this basically relies on the instances having already happened. And sometimes something might happen so infrequently

16:33 that by the time you catch it, it’s too late. And you still might want to quantify that as an error, but it doesn’t do you as good as if you’d known about it ahead of time. And that’s really the idea behind red teaming, which we launched recently. The idea behind red teaming is to help find these issues proactively. And so there’s a bunch of different ways that we think about doing this. There’s a bunch of different dimensions that we look at. So one is just the overall correctness of the response. Can we find issues where the agent is just not correct in how it responds?

17:04 The challenge here is how do we know if it’s correct or not? So what I mean by this is in order to judge whether an answer is correct or not is actually not trivial at all. That requires a ton of domain knowledge. If you could do that reliably as part of the agent loop, then you just do it as part of the agent loop and run it and then judge it. And then if it says it’s incorrect, just keep on running it. And so having this universal judge for correctness is one of the things that makes this hard to do. We haven’t solved that.

17:34 We don’t have a universal judge for correctness. But what we do is we have a few different algorithms that probe the agent in clever ways so that we do know what the correct answer should be. So for example, one of the algorithms that we have is we basically send two very similar queries, at least semantically similar queries, to the agent. And if it responds in semantically different ways, we know one of them is wrong. And then we can investigate which one is more likely to be right or more likely to be wrong. And so by doing that, we can ground basically the answer in

18:07 what we know the right answer to be. It’s not something that you could do in the loop, because you’re probably not going to do that as the agent’s running. That would be too crazy and too expensive. But it provides this kind of reference point. There’s other types of issues. There’s types of issues that are related to security, that are related to kind of like fairness. In the long run, what we want to have is basically a menu of these different issues that you can check for, and that they can generate these basically reports, these red teaming reports on the fly. One of the things that we’re also marrying with this,

18:38 I mentioned kind of like one of the issues with running evals is this simulated environment. You’ll need that for this because this does run the agent. And so we’re also doing a lot of work on basically creating these simulated environments. And our labs team in particular is doing a lot of research there. So at a high level, this is like the really high level abstract kind of like flywheel that you want to set up in order to improve your agent over time. You want to run your agent. You want to get feedback on how it does, either by a human or by an online judge.

19:09 You then want to go in. You want to see what happened inside of that trace to get that feedback. You want to add that as an eval data point so that as you try to improve, you can make sure that you’re actually improving on that and not regressing against previous ones. And then you want to suggest changes to the agent, either update to the model weight, update to the harness code, or update to the context. The really kind of future version of this is where all of this is done by the agent itself. And so the agent notices that it made a mistake. The agent goes, creates an eval data point of its own,

19:39 suggests its own changes, and it basically generates its own fixes, again, either to the context or the harness code or the model weights. We’re starting to think about what this looks like in Engine. This is very much a North Star vision. There’s a ton of nuance in this loop as well. Again, this loop is very high level, so there’s a ton of nuance there. We spend a ton of time thinking about the nuance. If you’re thinking about how to do this and want to dive deep into the nuance with us, please talk to us about this loop. We get really excited by this.

20:09 So that’s all I got for this talk. So thank you guys for the time.