The OpenSource Managed Agents

Author: Sam Witteveen · Watch on YouTube

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: The OpenSource Managed Agents · Channel: Sam Witteveen · 20:16 · Watch on YouTube

Topics: AI agents, Local and open-source models, Open source and software ecosystems, Privacy and data protection

Summary

The video introduces TrueForge, an open-source, MIT-licensed harness for running managed agents locally or in a private cloud with different models, tools, skills, and MCP servers. It contrasts coding harnesses, build-it-yourself frameworks, and hosted managed agents, arguing that the harness itself can strongly affect token use and costs, while hosted systems may create vendor lock-in and compliance concerns. The presenter demonstrates setting up models, connectors, and skills, creating an AI-news agent, and calling it through a Python client. He concludes that TrueForge is worth exploring, while noting that local sandbox execution is a desired roadmap feature.

Key points

TrueForge is presented as an open-source, MIT-licensed alternative to provider-hosted managed agent runtimes, designed to run locally or in a private cloud.
The video distinguishes coding harnesses, assemble-it-yourself agent frameworks, and hosted runtime harnesses such as managed agents.
A harness controls tool calls and context-window contents, which can significantly affect agent costs even when using the same model.
A benchmark described in the video reports that TrueForge used fewer tokens than Claude Managed Agents on the tested tasks, with comparable accuracy.
Running an agent in a self-managed environment can offer greater control over models, data handling, and deployment requirements.
TrueForge supports configuring models, connectors, skills, and MCP servers; the demo uses a DeepSeek model and Tavily.
The chat UI is framed as a test bench; agents can also be accessed through an HTTP API or TypeScript SDK.
The presenter builds an AI-news agent and demonstrates calling the locally hosted agent from a small Python project.
The presenter says local sandbox execution is a key feature he would like to see added, and notes that it appears on the roadmap.

Chapters

Start	Chapter	Summary
00:00	Introducing TrueForge and agent harnesses	The presenter introduces TrueForge and distinguishes coding harnesses, do-it-yourself frameworks, and hosted managed-agent runtimes.
04:18	Harness costs, control, and lock-in	The video explains how harness design affects token costs and discusses the benefits of avoiding provider lock-in.
09:07	Local setup and configuration	The presenter describes TrueForge’s local setup and configures a model, connectors, and a research skill.
13:24	Testing and building an AI-news agent	The demo tests the agent setup, creates an AI-news workflow, and saves it with its instructions and settings.
17:12	Calling the agent and roadmap	The presenter calls the agent from Python, reviews TrueForge’s roadmap, and shares his overall recommendation.

Quotes

Building an agent is easy, but running one well is not. — 03:43

same model, same answers for roughly a third of the tokens — 06:27

the harness, not the model, is usually what’s driving your agent costs. — 07:00

The chat interface there is the test bench. — 09:39

Concepts and entities

TrueForge, TrueFoundry, Sam Witteveen, Claude Managed Agents, Gemini Managed Agents, LangGraph, DeepSeek V4.1 Flash, Tavily, Ollama, LM Studio

Full transcript (original language)

00:00 Okay, so vendor-neutral, MIT licensed, on your own hardware without a runtime bill. These are all things that Claude Managed Agents doesn’t have. also the new OpenAI Agent API or Gemini Agents doesn’t have them. But all of these things are the things that the agent system I’m going to show you today actually has. Today, I want to talk about a way to run managed agents fully locally or in your own private cloud with any model that you want, adding in any tools that you

00:32 want, skills that you want, MCPs that you want, and being able to do it all really easy without you having to go into low-level code and tinker with every tool call or every skill that you add. Today I wanna talk about a self-managed agent execution harness called Trueforge. Now, before we get into it, this video is sponsored by TrueFoundry, the creators of TrueForge. but everything I’m showing you today is in an open source repo. There’s no paid tier in anything that you’ll see.

01:02 And the ask from them is basically that if you find this useful and interesting and you would like to see them work on it more, just go and star the repo. That’s it. I worked with TrueFoundry earlier on in the year and made a video about their Gateway platform, and it seems to me that as they’re developing things for enterprise, they’re also looking at how they can make some of these cool tools available to the open source community as well. In this video, I want to try and untangle that a little bit. I wanna talk about how you should be thinking about the different kinds

01:32 of harnesses that are out there and how you can be using them for your different kinds of projects. So let’s start with the word because I think this is where a lot of the confusion is. When most people hear the words agent harness right now, they tend to think of coding harnesses. That’s things like, Claude Code, Codex, Py, OMP, OpenCode, a whole bunch of these. And those are harnesses, right? They wrap a model, they give it a file system, a shell, some tools, and they run the loop for you. But they’re really built for one job, right?

02:04 which is writing code. The second kind of harness that’s out there at the moment is what I would call the assemble-it-yourself framework. So this is LangGraph Deep Agents, perhaps Agent SDK or ADK. And these give you all the pieces, and you write a loop, and you wire up everything yourself, right? You decide how the state gets persisted, how approvals work, and so on. And I’ve done quite a number of videos on those over the years. But nowadays it can be actually really tedious to use a lot of those frameworks if you’re trying to build your own harness.

02:36 The third kind and the one that’s most recent is one I actually just made a video about recently, which is the whole idea of a hosted runtime harness. So this is what people are calling managed agents. So you’ve got Claude managed agents, you’ve got Gemini managed agents, and I made a whole video about this, talking about the different kinds of managed agents that are out there. But the whole idea with those is that you define an agent, give it a model, almost certainly it’s going to have to be the provider’s model. You’ll have to give it tools, which again will be often provided

03:09 for you, and a system prompt. and then that company’s gonna run the loop. So session, tool calls, sandboxes, approval, all of that is handled. You just hit an API. And that third category can be really nice to use, right? I’ve built with it, et cetera. But here’s the thing. Up until pretty recently, that category really meant you only had one choice in that it has to be hosted with the actual provider. And perhaps even worse than that, You’re often just tied down to certain models, that may just be not a great fit for what you’re trying to do.

03:43 So TrueForge is an open source version of that third kind And their README puts it really nicely: “Building an agent is easy, but running one well is not.” The running it well part is all about streaming, session persistence, tool servers, sandboxes, approvals, a UI. That’s all the stuff that really honestly most people don’t want to write, and it’s the stuff that managed agents have been charging you a premium for. Trueforge is that layer as a repo that’s fully open source, MIT licensed,

04:18 that you can clone and then run with any particular model that you want to. Alright. So the question should be, why does it matter what kind of managed agent I’m using? And this is the bit that I think many people, even myself at times, have gotten wrong, that you tend to think of the cost of all of this as just being related to which model that you’re using and how much you’re paying per token. And then often you don’t think about the harness and all the plumbing that is actually going on around this. But when you start running managed agents at volume, you start to realize, that

04:52 actually that harness really is often the key thing because it’s that harness that is actually deciding how many tools calls you make, what tool calls you make in what order, how it’s going to actually decide what goes in the context window. Remember, really what we’re paying for is whatever’s in the context window. So if you’ve got lots of tool calls in there, if you’ve got tool bloat, right? Where you’ve just got so many tool definitions and stuff in there, and you’re not using progressive disclosure and things like that, you’re gonna

05:24 run up a much bigger bill than you would with a very efficient harness. And this is where the True Foundry team actually put out a benchmark that shows all of this. They took 14 tasks from DevRev’s Enterprise Bench. The kind where the agent has to join data across a CRM, an issue tracker, a document store, multiple, MCP servers, all with the same prompt, and they ran three trials each and used a blind LLM judge. And here’s the thing, they didn’t even go for cheaper models.

05:56 They actually ran it on Opus through three different harnesses And you can see from this diagram here they basically showed with the actual TrueForge harness that even using the same model, they were able to get the cost down by 30% with the same amount of accuracy. and that’s because Claude Managed Agents was using 10 million tokens per run on average versus only 3.8 million tokens on the TrueForge. And of course, when they moved that to a model like GLM 5.2,

06:27 suddenly the model cost came down. The whole thing now was down 75% compared to the original Claude Managed Agents version. So even if we skip the GLM 5.2 and just look at, the same model, same answers for roughly a third of the tokens, that was all Harness decided. So the fact that TrueForge is actually open source means that you can pick any model that you want here. You can decide if you want to go for a more expensive, Claude model or OpenAI model, et cetera.

07:00 But you can also go for much cheaper open models, like I’m going to show you in here. So if you just take one thing away from this video, it’s this: the harness, not the model, is usually what’s driving your agent costs. And the harness is the part that you’ve actually got a lot of control over, even if you’re not running your own models with your own GPUs, et cetera. Now for me, I would say while the cost savings here are the headline, for me the lock-in thing is probably a bigger deal than a lot of people are watching.

07:30 just in the past week we’ve seen a number of different news items about what user data these people are actually training on and how they’re actually using user data. And you’ve got things like data retention. if you want to have zero-day data retention, you cannot use Claude managed agents. If your agent definition lives inside a hosted runtime, then the system prompt, the tool wiring, the approval rules, the session history, all of that sits on someone else’s product. And perhaps that’s fine up until the day you wanna try a different

08:03 model, or you wanna try a different sandbox, or you want to try a different configuration, et cetera. You don’t want to get locked into these services where you become dependent on them for a lot of things beyond the model. We’re already seeing, open models get so good that really high levels of intelligence are now being democratized so you can access it over lots of different providers. Even if you have objections or legal things around sending your data to certain countries, the cool thing now is with open weights models, there are providers

08:35 popping up everywhere that are running the same weights with the same API endpoints And I hear this a lot from people building inside larger companies. They often have a mandate to run things on-prem, or at least inside their own cloud account. And a hosted agent system just isn’t gonna get through procurement, through compliance, no matter how good it is. So they fall back into a category of trying to write the entire loop themselves. And that can also become both an inefficient and expensive option. And then of course, if you do have some local hardware that you can run some

09:07 local models on, we’re already seeing the smaller models get better and better, And this whole system allows you to integrate with things like Ollama, with things like LM Studio to run models locally. All right, so TrueForge is a MIT license TypeScript repo from the True Foundry team. With one MPX command, you can basically spin this up. It takes care of a local SQLite database. All your data is staying locally, and it’s running on a local host. There are also hosted modes If you want to do it with things

09:39 like Postgres or Redis and Docker. but the whole local mode is a very deliberate step on Truefoundry’s path to make this as easy as possible for you to get started trying it out. Now they have a whole chat UI, but don’t be fooled into thinking that this is another chatbot. It’s not. The chat interface there is the test bench. It’s where you poke around and play with the agent and test things out before you actually save it. The actual main thing here is the runtime and it’s actually exposed a few different ways. You’ve got the UI, that you can start off with, but you’ve also got an

10:11 HTTP API endpoint, and then also you can use that with a TypeScript SDK. This means that you can build something in this very quickly, expose the API endpoint, and then have other apps ping it. So this makes it ideal if you’ve got people on your team, for example, that you want them to basically run different agent workflows, but you don’t want them having to worry about how to configure the tools, how to actually prompt it right, how to do all of that.

10:43 You can actually just bake this into something like Slack, a mobile app, any sort of software that you want it to write. So let’s jump in and have a look at how this actually is all put together Alright. So to get started here, this is pretty simple. We’re just gonna go, to the ReadMe docs and see, go to the quick start, and Then I’m gonna just install this locally, to get started And you can see this, this will basically install all the components in here. Everything from, being able to deal with sandboxes, MCPs, skills, all those

11:15 sorts of things And it’s actually now running locally so that I can get started Okay. So once you’re in the actual interface in here, you can come into the settings and you can set up models, connectors, skills, and sandboxes. So models here, you’ve got your standard, sort of off-the-shelf models. If you do want to use Claude, you can use Claude. If you want to use Gemini, you can use Gemini, et cetera. you’ve also got a bunch of, open providers, Or you can configure something totally private. Here I’ve configured up my own API endpoint to run DeepSeek Flash 4.1.

11:50 So this model is just out the previous 4 Flash was one of my favorite models, So I think like a lot of people, I’m quite excited to see how this new model goes. But you could configure lots of different models in here. Next up, you’ve got connectors. So you can see I’ve already gone and configured, Tavily, Which is a really good search tool. you can get a free API call. I think you get 1,000 calls a month or something for free. also DeepWiki, is one of the ones that I like. but there are lots of others in here. So if I wanted to add my Notion, I can go ahead and do that. if I wanted to add Bright Data for doing any sort of, scraping or things

12:23 like that, I’ve got that in here. Or I can just come in here and totally configure up, a personal MCP server. So that could be running locally, that could be running in the cloud, et cetera. And then you’ve got skills. Now skills is obviously one of the best ways to actually get started. So I’m gonna put in the Tavily, research skill there. So I’m gonna enable that so that the folder contains the skill.md, I can then just import that and use that as well. And then lastly, we’ve got, sandbox providers. So they’re a bit light on this at the moment.

12:53 I think TrueFoundry’s also got some interesting things around sandboxes that you might see coming in the near future. But once I’ve got this set up, you can see I can just come along and I can do a new chat. Now, the chat ID here is really just to allow you to test that everything’s connected up and stuff like that. So you can see I’ve got my DeepSeek, v4.1 flash model in there. I’ve put on Tavily as a tool in there And if I wanted to import some files or something like that, I can use this

13:24 just like a normal chat interface. But it’s very important that you understand that this is not the product, right? This is just to basically test the product. so the idea here is that, okay, now I can see that, okay, everything looks like it’s running. I’m getting reasoning tokens back from 4.1 Flash. I can see that, it’s running through the different tests, and we’ve got the ability now to, run and see what actually is going on here. you can see that the agent has gone off and just me saying that this is

13:55 basically a test to, wire things up. I actually was just planning on testing, the actual LLM, but it’s gone through the whole system, and this is the cool thing of this, right? All of these things are now integrated as an agent. So it’s acting as an agent with these tool steps and with this thinking. So it’s taken this as like my task of that, “Hey, we want to check that everything’s set up.” It’s gone through, it says, “Right. Yep, we’ve got the date, time. We’ve got the, MCP set up. We’ve got our tool execution.

14:25 Everything on that is working.” we can see it failed on a sandbox because I haven’t set up a sandbox yet, right? So, It’s good to know that this is already starting to work Okay, so if I want to build a new, agent, let’s say I want to build an agent that’s gonna go and get AI news for me. So I’m just gonna do a very quick one here just to sort of show you how you would do it Okay, so you can see I’ve got my, prompt there, and what I’m doing here is basically just going through this, letting it work out, the different things to do.

14:58 And the cool thing is, it’s doing all this in the background. It’s using Tavily. I didn’t need to tell it to do that. It basically worked out to do that. It’s also then gone to the VentureBeat site, which is the link that I gave it, worked out that, okay, we probably want the category AI, in there. Now I don’t have a sandbox set up, so it’s obviously working out that that’s gonna be an issue But you can see very quickly it’s basically worked out itself what it needs to do, like extracting the homepage, trying out different things, et cetera, for doing this And then the cool thing is it’s then used Tavilly to actually do searches,

15:31 passing in info knowing that I’m just looking for the last 48 hours We can see, what it’s actually done there with, Tavily And you can see that finally, it goes through, it finds out like the DeepSeek, V4.1 Flash release, different releases that have come out, ChatGPT for financial services, et cetera. And then even for other ones, It’s then focused on checking the date, right? To see if those are right. Now, once we’ve got that, we can then come over here to actually save this. I then basically put in my, instructions, which is like my

16:03 system prompt here, for doing this. I can add any skills that I want. So I’m gonna put in the actual Tavily research, skill in there And then I’ve got to decide some things like, okay, do I want the agent to ask clarifying questions? Do I want it to have dynamic sub-agents? Do I want it to have basically, like return a generative UI? Actually, I don’t want the generative UI or the clarifying questions on this. I just want to sort of ping it and basically have it return the result. So now I can save the changes, and it will go through and create this agent for me.

16:39 And you can see Now if I want to basically just try this out, I can come in here and put in, the prompt to kick it off, and you can see sure enough, it’s going off. Even though I wrote, “Just get me the news,” it understands that, this is gonna be AI news. I actually changed it to be, 72 hours in the system prompt there. it’s got today’s date, it’s got the different, parts that it’s going to basically get there. And you can see that already it’s got a bunch of sort of the recent, stories that have come up And so now it’s gonna basically check the dates of those, And All of this is

17:12 running with the DeepSeek model, right? So this is a very cheap model, to run And you can see that its final output is coming back with the actual stories, with the summary, and with the, the source, URL for each of these. so this is pretty good, right? If I check something like this sure enough, this was written in the past day or so. and then It’s also putting some honorable mentions there Okay, so if I want to use this with Python, I can actually just come in here and basically set up a, a little UV project with just a single Python file.

17:43 if we look at this, it’s just going to basically call the, local host version of this. This is where the agent is currently being hosted. You could actually have it, running, in the cloud if you wanted to do that, obviously. I think going forward you’ll see True Foundry have their own sort of managed versions of this, which lets you prototype everything locally and then ship it to their cloud to run regularly at scale kind of thing But you can see if I wanna basically run this now, all I have to do is UV run main.py.

18:13 it will basically set it up. It will go through, setting up the different tools, et cetera, and then it will go off and do the task and here I’m just streaming all the things back. But if you wanted to, you could get it to basically just give you an output file or to save to a database or something like that at the end And you can see now that it’s finished off here, we’ve got all the results back. we’ve got the, OpenAI launches financial services. We’ve got DeepSeek, v4.1 Flash in there.

18:43 And for each of these, we’ve got the VentureBeat article. We’ve got, the date. We’ve got the different information, in there So just to finish up, I would say This is definitely worth checking out if you wanted to run agents, locally, on your own machine, et cetera. and the cool thing is once you’ve actually created these agents, they’re stored there, so you could actually install this in the cloud and import them in there and use that Now looking at their roadmap, it looks like they’ve got a bunch of things coming up. you can see that they’ve planned to have a gateway integration, which totally makes sense.

19:14 TrueFoundry has a lot of sort of gateway features, both for models, for MCPs, for locking things down like that. It looks like they’re addressing probably my biggest issue with it is that I would like them to have a local sandbox execution, and it looks like that’s coming. It’d be great if you could also lock that down with Docker sandboxes or something like it just to make it fully safe and secure and looking at the rest of the roadmap, It all seems very sensible stuff if you wanted to do this kind of thing. And a lot of it is perhaps stuff that is in the lockdown versions of

19:44 managed agents So it’s great to see that this is all coming in the near future So go give it a star on GitHub. The links are in the description. Really, this is giving you all the advantages of a managed solution without the vendor lock-in and without the downsides, et cetera so let me know in the comments, what are the biggest features you would like to see added to this? Talking to the team at True Foundry, They do seem very committed to making this work, both for their customers who are using it, but also for the general open source community.

20:16 Anyway, as always, if you found the video useful, please click like and subscribe, and I will talk to you in the next video. Bye for now.