Inside Managed Deep Agents: mda CLI, Context Hub, and Harbor Evals | Interrupt London

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: Inside Managed Deep Agents: mda CLI, Context Hub, and Harbor Evals | Interrupt London · Channel: LangChain · 14:42 · Watch on YouTube

Topics: AI agents, AI evaluation and benchmarks, Prompt and context engineering

Summary

Victor introduces Managed Deep Agents as a way to take domain-specific agents from local prototypes into production by combining a customizable agent harness with production infrastructure. The Deep Agents harness provides an execution environment, context management, delegation, and human oversight, while Managed Deep Agents adds capabilities such as durable execution, authentication, memory, and safe code execution. The talk also covers the mda CLI, Context Hub for collaborating on agent context, and Harbor integration for setting up evals. Victor closes by highlighting design partners and recent features, including custom HTTP channels, agent-created schedules, and per-run configuration.

Key points

LangChain describes an agent as a model plus a harness: the skills, memory, system prompt, and other components that provide the right context at the right time.
The Deep Agents harness centers on four areas: a file-system-based execution environment, context management, delegation to subagents, and human-in-the-loop steering.
Madrigal built a domain-specific research harness by codifying expertise into skills, with its platform team helping create evals.
Production agents need infrastructure for multi-tenancy, persistence, durable execution, streaming, and task queues; Managed Deep Agents integrates this with the harness.
The mda CLI scaffolds projects with mda init, supports local development with mda dev, and deploys agents with mda deploy.
Agent projects can define their loop, instructions, skills, tools, authentication and identity, and memory in TypeScript or Python.
Managed Deep Agents supports user- or agent-scoped credentials and can pause a run for authorization, then resume it later using durable execution.
Context Hub provides a shared, version-controlled space for business users and developers to view, edit, and collaborate on agent context and skills.
The Harbor integration helps configure eval environments and checks, and the mda evals init command can use a coding agent to assist with setup.
Recent additions include custom HTTP channels, agent-created schedules, and per-run configuration for dynamically creating agents.

Chapters

Start	Chapter	Summary
00:06	Agents and the Deep Agents harness	Victor defines agents as models paired with a harness and introduces the execution, context, delegation, and human-oversight capabilities of Deep Agents.
03:14	From domain expertise to production infrastructure	Using Madrigal as an example, Victor explains how teams encode expertise into skills and why production agents need infrastructure alongside business logic and a harness.
05:21	Managed Deep Agents and the mda CLI	Victor describes the managed runtime and integrations, then demonstrates how the mda CLI scaffolds, runs, and deploys an agent project.
08:27	Authentication, permissions, and memory	The talk explains how agents identify callers, use user- or agent-scoped credentials, pause for authorization, and manage personal or shared memory.
11:34	Context Hub, Harbor evals, and recent features	Victor covers collaborative context editing, Harbor eval setup, design partners, and newer features such as custom channels, schedules, and per-run configuration.

Quotes

models are only as powerful as the context that they’re given. — 01:08

which is the business logic. — 05:21

we can also pause and wait indefinitely for access to certain systems. — 10:02

evals are very, very easy in Managed Deep Agents. — 12:06

Concepts and entities

LangChain, Managed Deep Agents, Deep Agents, agent harness, LangSmith Deployment, LangSmith Sandboxes, Context Hub, Harbor, LangGraph, Madrigal, Parallel, Kyth, Consensus

Full transcript (original language)

00:06 Alrighty. Hello, everyone. I’m Victor. I’m a product manager here at LangChain. And today we’re going to be talking about deploying mission-critical agents into production, and sort of everything that it takes to take a deep agent, for example, and bring it into production with Managed Deep Agents. So the first place that we like to start, and it’s a question we often get, is: what is an agent? Harrison talked about this a bit earlier today. The first definition that we sort of had at LangChain was this idea of a ReAct-style agent, or a reason and action style agent.

00:38 And this is sort of an agent that can essentially react, use a set of tools, and reason and act over a particular task in order to accomplish that goal. What has sort of happened as of late is that’s evolved into this idea of an agent harness. So we’re now going into this model plus harness, and that’s sort of what defines an agent for us. And this is everything that we put around our model itself, right? So the skills, memory, system prompt, all of those things.

01:08 The question that we get a lot is: why do we go through the effort of this? Why do we go through the task of building out an agent harness specific for our use case? And it’s really in order to get the model the correct set of context at the correct time in order to do its particular task. And we truly believe that models are only as powerful as the context that they’re given. And so when we start to build harnesses ourselves, we really wanted to start focusing on building a customizable sort of base harness. And that’s what Deep Agents is. It’s a great way to get started and build

01:41 out that domain-specific harness for your use case. So what kind of goes into this Deep Agents harness? So there’s sort of four key areas. The first being the execution environment, and this is really the backbone of Deep Agents: the idea that you can have an agent use a file system. So models are very, very good at using file systems now, writing code, things of that nature, and we really want to take advantage of that when we’re creating our agents. So Deep Agents has first-class support for that, and we can also pair it with a sandbox so that it can sort of execute that code in a safe

02:13 environment. The second core area is context management. So context management is very important through progressive disclosure of skills. We need to be able to manage memory and have the agent manage memory itself. In some cases, we want to be able to summarize conversations, compact them down, and offload context when needed. So if a tool, for example, returns a really large output, we can sort of store that in our file system and wait for the model to sort of reference that context when it needs it. The third key area is delegation.

02:43 So being able to decompose a task and delegate it to either specialized or general-purpose subagents is very, very important. And lastly is steering through human in the loop. And that’s something that is core to what we do here at LangChain. And part of why this conference is named Interrupt is that function to be able to encourage human in the loop and allow your agent to wait for a human to approve an action. So as we think about building out these domain-specific harnesses, one of the companies that I’m excited to highlight, as I’ve worked with them a decent amount, is Madrigal.

03:14 It’s a pharmaceutical research company, and they were able to build a domain-specific harness to help their PhDs, their researchers, really improve and standardize how they’re doing their research. So they have a variety of different areas of work that they do research on in the pharma industry, and they really wanted to codify this and make it really valuable for the rest of the organization to come in and build agents for their use case and build research agents for their particular domain. And the way that they did this was they broke these down, and they broke down their expertise into skills.

03:45 Their platform team and their engineers help them create evals and really start to spin that flywheel that I’m sure you’ve heard a lot about today. So when you think about building out an agent, something that Harrison sort of touched on a bit earlier today is it really breaks down into three layers. The first is the business logic, and this is what you bring, right? This is your expertise. This is what you need to build agents for yourself. The second is the harness. So we need somewhere for that expertise to execute, right? We need a system to execute it. And that’s sort of what this harness, or this Deep Agents base harness, provides.

04:17 And the third is going to be the infrastructure. So this is all the stuff around bringing an agent, making it multi-tenant, giving it a production runtime. Each one of these layers, all three of these, sort of really need to work together in a cohesive fashion in order to take an agent into production reliably. So we talk about taking an agent into production. The first step usually looks like this. We’re going to prototype it locally. Getting it running locally is sort of a magical experience, right? We can sort of prove that an agent can do a certain task for us. And that’s a very exciting moment. It’s a very valuable moment.

04:47 This is where we sort of prove, as developers, as agent engineers, that we can build an agent to sort of do something that is domain-specific. Then we start to go into production, and this is where things start to get a little bit messy. We need to be able to handle multi-tenancy, we need to be able to handle persistence, we need durable execution, we need to handle streaming. All of these different things that go into taking an agent into production start to become things that your team has to own if you’re doing this from scratch, right? And there’s a lot of different layers to this that can break down. So, for example, a task queue can be something that

05:21 can be tricky. You need to be able to prioritize which tasks are going to get picked up by the agent, what priority that you’re going to execute those. That all becomes tricky as you scale to many, many, many users. What we’ve done with Managed Deep Agents now, and why we’re so excited about it, is we’ve sort of taken that execution environment, the harness itself, and we’ve tightly coupled it together with the infrastructure. And this is a really important step in terms of making it really easy for you and your developers to go and work on what actually makes the agent better,

05:51 which is the business logic. When we talk about taking an agent into production with Managed Deep Agents, this is sort of how it looks. So all of those things, all those connections that you had to maintain as a team before by managing the harness and the infrastructure, things like durable execution, things like the task queue that I mentioned, these all come out of the box with Managed Deep Agents, as it runs on LangSmith Deployment, which is a proven production runtime that we have here. You get MCP and A2A in terms of doing your sort of orchestration across

06:22 many agents. We have Context Hub as a great surface for you to interact with business users on particular business logic and other things, such as instructions and skills. You can sort of collaborate with them in that Context Hub. We have LangSmith Sandboxes for safe code execution. This is very important when you think about giving your agent access to write and run code. You need to give it a safe environment to do so. Then we have our layers of governance with LLM Gateway and our tracing and evals, of course.

06:54 Now, since we’re handling a lot of this infrastructure with the harness, we’re able to sort of scaffold this project into a very coherent project structure. And this has actually paid dividends internally here at LangChain. We have a lot of developers and agent engineers that work across different agents internally. And having our agents defined in this project structure has now been very valuable in terms of being able to jump into any agent project and understand exactly how it works. So the way that we scaffold it is through one simple command. We can do mda init and give it a name like GTM agent, and that will go ahead

07:25 and scaffold this project structure for us. Once we define our business logic, we can then run it locally with LangSmith Studio using mda dev. And once we’re ready to go into production, we can simply run mda deploy and deploy it into production. So, walking through what this actually looks like, I’d love to use the example of a GTM agent that Harrison mentioned earlier today. This is a very powerful agent. Our sellers and our GTM team and people, including myself, absolutely love using it. It really helps on the day-to-day, keeping track of everything

07:56 that can go on in the workplace. But everything that goes into the deep agent now can be reflected for GTM agent into this definition. So the agent.ts or agent.py, you can do TypeScript or Python, will define your actual agent loop. The instructions will define your agent logic, sort of that core definition of how you want it to behave. We can then give it sort of like playbooks. For GTM agent, for example, we give it certain playbooks for different actions through skills, and then we can also connect it to all the systems that we need to with tools, connecting

08:27 to MCP servers, things of that nature. And then the last two are pretty important in terms of defining how we want to handle auth and handle identity, as well as handle what it remembers and how it remembers. So setting up this agent is an exciting part, and getting it all ready to go. You start to think about this next layer, right? And one of the hardest parts of building agents and putting them into production is ensuring that it gets access to the right context and it has the right permissions to get access to that context or these different tools.

08:59 So the first kind of key layer that we sort of start to see is we need to figure out who can call the agent, right? And the way that we do this in Managed Deep Agents is we have a centralized sort of like auth handler service that, depending on the channel or where you’re calling this agent from, we will go ahead and make sure that we establish who the principal is for that call, whether it’s the agent itself or a particular user, and we’ll go ahead and pass that along into the runtime context. So you’re able to identify the user, and you’re able to make sure that gets passed into different tools or other connections.

09:31 As we start to go into… We have that user’s context, and now as the run is going through, we need to see, okay, what account should the user use, or should the agent use, rather? In some cases, you might want to always ensure that the agent is only using user-scoped credentials. As a developer, you can define that very easily, as you see here. In other cases, you might want to have a service account or an agent-scoped credential, and that’s also very easy. You can see there’s just simply two types that you define as a developer in order to define that when you’re using tools or MCP servers, things

10:02 like that. One thing that we’re able to do as well, since Deep Agents runs on the LangGraph runtime, we have that element of durable execution, checkpointing, human in the loop, is we can also pause and wait indefinitely for access to certain systems. So, as a user, the first time that I use GTM agent in Slack, for example, if I need to authenticate with it to use our CRM on my behalf, it’ll ask me to go ahead and authorize, will send me a message only I can see. I can then come back three days later and say, “Oh yeah, I forgot I needed to do that.” Go ahead and hit authorize, and it’ll continue the run exactly

10:34 from where it left off. So because we have that durable execution, it makes it very easy to do things mid-task. The next portion is memory. So memory really helps us compound and learn over time. And there’s certain scenarios where you want an agent to be able to take sort of like agent-level memory to improve its procedural memory, or you want it to take user memory: personal preferences, how I like to write emails, how I like to reach out to customers, things of that nature. You’re able to define that very easily in Managed Deep Agents as well.

11:04 What’s really nice, since we’re managing this in code and we’re able to define this by the channel that it’s calling it, as well as the particular tool itself, I can control, based on where I call the agent, whether it’s going to use a set of personal memories, personal credentials, let’s say in a Slack DM, for example. And then if I want to use this agent also in a shared Slack channel, we also don’t want to leak any private information, of course. So we can have our agent use agent-scoped credentials or service accounts in those scenarios so that we know that it’s safe.

11:34 As I mentioned before, we have a first-class integration with LangSmith’s Context Hub, and that allows us to collaborate with our business users on the different context that your agent has access to. This is really valuable in terms of having version control over this context in a shared space that we can all sort of visualize it, interact with it, edit it. And one of the nice things that we see is people often collaborate on skills. They can promote skills to workspaces, things of that nature. And once you start making these changes, one thing that we know is very valuable in terms of working with a lot of customers, especially customers

12:06 that use LangSmith, is evals. So evals are very, very easy in Managed Deep Agents. We have a first-class integration with Harbor, which makes it really quite easy to set up all of these sort of intricate details that Harbor allows for, in a very managed process. So I can run mda evals init, and if I pass this interactive flag, it’ll actually run with a coding agent and help you set up the Harbor environments, set up the checks, all these different things that go into building really good evals. And then you can start to iterate and hill climb on those

12:37 particular set of eval datasets and experiments. Something we’re really excited about is we launched a partnership with Parallel, who you can find in the demo stations, and we are offering zero data retention, and you can essentially use it directly through LangSmith without having to go get your own Parallel search key. We think that this is really important. Having access to really good data through different search APIs is very valuable for most agentic use cases. You can really easily add it to your managed deep agent via MCP, as such, and

13:08 access it through LangChain itself or LangSmith. We’re really excited to highlight a few of our design partners that were with us from the very beginning of working on this project. So on the left side is Zahid, who worked with us from Kyth, and really helped us define how we did user-scoped credentials, memories, things of that nature. He was very opinionated as well, which is very helpful in a design partner. The second would be Consensus, which Derek and his team had a really cool use case of actually going around as a tiger team into different

13:39 parts of the org and building out these internal agents for different parts of the organization. And that’s something that we’re seeing become more and more popular as customers start to sort of expand the sort of breadth of internal agents at their companies and providing that expertise for other teams. So what’s new in Managed Deep Agents as of this past week is all of these wonderful things. There’s a few different ones that I’d like to highlight. The very first one, HTTP channels allow you to define custom channels. So we have a first-class integration with Slack, as well as GitHub and a

14:11 few others. But we also wanted to define a flexible spec for you as developers to go ahead and implement your own channels and how you want to use your agent. And then towards the bottom here, agent-created schedules are really nice. You can sort of tell an agent, “Hey, every Monday at 9:00 a.m., run this report for me, or go and find what customer meetings I have,” things of that nature, and it can go ahead and create its own schedules. And then lastly, the per-run config allows for you to create dynamic agents. So depending on the context of where I call the agent, I can insert some

14:42 runtime configuration and sort of dynamically create that agent on the fly. And those are both very, very powerful. Awesome. So we’re excited to see you all try out Managed Deep Agents and bring your domain expertise, and can’t wait to see what you build. Thank you.