Let's Learn GitHub Copilot SDK

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: Let’s Learn GitHub Copilot SDK · Channel: GitHub · 1:32:27 · Watch on YouTube

Topics: AI agents, Software engineering with AI, Model inference and optimization

Summary

The video introduces the GitHub Copilot SDK, its evolution from the CLI SDK into a portable runtime SDK, and the range of applications it can support. The presenters explain how an agent runtime works and walk through configuring harness profiles, prompts, tools, context, custom agents, inference providers, policies, hooks, and session storage. A live demo adds an agent to a markdown editor from Tiny Tool Town and illustrates how the SDK can be integrated and customized. The session closes with hosting patterns, resources, and examples of how teams can build with the SDK.

Key points

The SDK reached version 1.0 in June and has since moved to version 2.0, with a Rust rewrite supporting a more flexible runtime.
The SDK supports six languages and is designed to work across local, cloud, and multi-tenant environments.
The runtime is an agent loop that combines context, model inference, tools, permissions, hooks, and session events.
Integrators can choose a base profile, customize system prompts, and add, remove, or override tools.
Context and configuration can be discovered from locations such as repositories and home folders, with additional skill-provider options in development.
The SDK supports custom agents, root-agent selection, and tool restrictions to focus agents on specific responsibilities.
Inference can use GitHub Copilot or external providers, with options for local models and configurable credentials.
Policy handlers and hooks let integrators inspect or block tool calls, observe activity, and govern execution.
Runtime hosting ranges from local and container-based setups to production architectures with runtime pools and supporting services.
The demo integrates the SDK into a Tiny Tool Town markdown editor, showing that an existing app can be enhanced with an agent.

Chapters

Start	Chapter	Summary
04:05	Opening caption	The video begins with a brief caption before the presenters start the session.
09:18	Welcome and agenda	Matt Soukup and Patrick introduce the session and outline the topics they will cover.
11:22	From SDK 1.0 to 2.0	The presenters describe the SDK’s release history, Rust rewrite, and support for six languages.
13:26	The runtime SDK and its uses	They explain the shift from the CLI SDK to the runtime SDK and discuss production uses across different kinds of applications.
17:00	Portability across environments	The conversation covers local, cloud, multi-tenant, and restricted environments that the runtime needs to support.
19:03	Visualization tool and Tiny Tool Town	Patrick introduces a tool for visualizing SDK configuration, and the presenters explore Tiny Tool Town as a source for their demo.
23:08	Adding an agent to a markdown editor	They use the Copilot app to ask an agent to integrate the SDK and add chat-based editing features to an existing app.
27:47	How the runtime loop works	Patrick explains how the runtime loop gathers context, accesses model providers and tools, handles permissions, and records session events.
33:30	Base profiles and harness modes	The presenters compare empty, minimal, and Copilot harness configurations and discuss choosing a starting profile.
40:10	Customizing prompts	They show how to use default system prompts, append instructions, or customize specific prompt sections.
48:31	Tools and integrations	The discussion covers built-in, custom, and MCP tools, including ways to add, remove, or replace tool behavior.
1:02:31	Context, skills, and custom agents	Patrick explains configuration discovery, skill locations, custom agents, root-agent selection, and limiting agent tools.
1:09:11	Inference providers and policies	The presenters discuss Copilot and bring-your-own inference options, followed by policy handlers and hooks for governing execution.
1:15:50	Session state and runtime events	They describe session storage options, large-output handling, streaming, and the runtime’s event-based architecture.
1:22:02	Build options and hosting architectures	The discussion covers language and process choices, in-process runtimes, and hosting patterns from local setups to production systems.
1:26:11	Demo results and resources	The presenters revisit the editor demo, share SDK resources and links, and close with a multi-agent interview-coach example.

Quotes

We’ve now moved on to 2.0 — 11:22

it must be extensible, it must be configurable, and it must be portable. — 18:01

The runtime at its core is a loop — 27:47

you can take that control — 39:08

you can actually run it directly in an application — 1:23:36

Concepts and entities

GitHub Copilot SDK, runtime, Harness, Agent Loop, Tiny Tool Town, MCP servers, Copilot app, Matt Soukup, Patrick Nicholas, GitHub

Full transcript (original language)

04:05 ICAP TEST CAPTION …RETURNING TO UPSTREAM…

09:18 Hey everybody. Welcome to Let’s Learn GitHub copilot SDK where you’re going to learn everything that you ever, ever wanted to know about the copilot SDK. My name is Matt Soukup and I am joined by my good friend. And by the end of this we will be best friends. Patrick. Nicholas. Patrick, tell us a little bit about yourself. How are you doing today? I’m doing fantastic. Thanks, Matt. Yes, I’m the product manager over the copilot SDK, previously typically known as the CLI SDK. But we’re going to talk a little bit about its transformation and lots of the

09:49 great capabilities that are coming in the near future. Great. Yeah. Should we take a look at the agenda? We dive right in. We have a lot to cover. Yeah. So in terms of what we’re going to cover today, we’re going to talk about the basics. What is the actual copilot SDK? Because I think there’s a lot of questions around what is it really meant for? What is it actually power and what can you do with it? We’ll explore the runtime. Beneath the SDK is the actual engine that drives our agent to harness itself. And so we’ll dive a little bit into the details there. It’ll really set you up for success,

10:20 understanding all the mechanics of what’s going on beneath the surface. I thought we would customize a harness. I think everyone is out there building harnesses all over the place. That’s also really interested in our copilot harness. And so very excited about that part. Wonderful, wonderful. And then what would a harness be without adding it to an app? So I think you made a great suggestion last time that we would take a look at our good friend Scott Hanselman, Tiny Tool Town, pick a tool and go ahead and add that in. And then there are closing resources, things you

10:50 can play with, definitely the code as well as a nice little tool we’ll be showing off that’ll educate, allow you to build harnesses and really dive deep, learn on your own as well. Great. Yeah, this is going to be a really fun session. I’m really looking forward to the demo part so we can really see the power of the SDK. Wonderful in action. How about we take a look about what is the SDK, what’s coming up? And then let’s dive right into the fun because there’s a lot to show off today and a lot of things we want to get through as well. The first thing to be aware of is we did GA the SDK in June of

11:22 this year. That was our 1.0 version. It was a really big delivery because it enabled a whole lot of teams to stick the SDK in their real applications and enabled us to support it in production. We’ve now moved on to 2.0, so we’ve made a lot of changes beneath the surface, essentially rewrote the whole thing in real time as we were adding new features to our runtime. All of our developer tooling, our great distinguished engineer on the team, Steven Taub, just published a post on the GitHub

11:53 blog. If you haven’t read it, you definitely should. It’s super long, super detailed fashion. It’s going to crash your browser, probably. Yeah, it it will. Yes. And so if you know anything about Tobe, he loves the tobe post, which is very detailed. He’s a great performance and runtime engineer. And there’s a whole host of reasons why we did it. But we’ll, we’ll get into the details around really the flexibility of SDK, all the places it needs to operate and why things like the rust rewrite were really essential to our future to in terms of

12:24 the basics, we are supporting six languages, which is fantastic. Whether you’re using TypeScript, whether you’re using Net go, Java, rust itself. We have support, we have SDKs available for all the languages. We keep them up to date. We keep them in parity as best we can. We want to make sure that you can use our runtime wherever you might be, in whatever language you’re in. And that was one of the things I really liked about it, is that it’s there for where you are right now. You don’t have to go out and learn rust in

12:55 order to use this, which, let’s face it, a lot of us won’t. Yes, yes. So I’m dot net and it’s there for me to use. Absolutely. From day one, everyone has their preferences. And I think what’s wonderful in the Atlantic area is you can either continue to go deep there, you can try new things, but we want to make sure as you explore and are using agents in a really diverse set of applications and platforms, the support is there at bare minimum, which is really critical and important. The other aspect that I touched on is our 1.0 was the CLI SDK, and I think most people still

13:26 believe we are the CLI SDK. And it was a wonderful era throughout the first half of this year. These are just three examples of things I was building with browser use plugins, my own custom office add ins, chat interfaces, all kinds of tools. I built a whole host of things and just about shoved the agent anywhere I could through that period, not only to educate myself and identify kind of the boundaries and limits, but also really highlight to end users and customers the range of

13:56 applications, what it means to take our harness and just put it into an application and supercharge it with our agent capabilities as well. But today, after a lot of collaboration internally, we now refer to it as the runtime SDK. Okay. Which, which I think is really important because we’ve dropped the runtime out of the CLI officially. And now we have this really great portable package that can be used with our SDK clients. And it’s really due to a lot of internal Cross-collaboration. Many of

14:27 the teams and organizations you even see on this slide have actually shipped our SDK inside of real production applications. So if you use things like Office Excel, there are new agent mode. The SDK actually sits at the core of that, which is just a wonderful opportunity to collaborate and diversify. That’s actually super cool that these huge, huge mission critical projects.for.net for Microsoft are using this wonderful today. It’s not only really amazing for us on the GitHub copilot team, we get to pressure test our abstractions,

14:59 how you can configure the runtime, how you can extend it. We get to dive deep into their domains, very different from coding software development domains. So you get a look across really a broad spectrum. There’s knowledge worker use cases, security use cases, creative use cases, etc. and we’ve really used this as an opportunity to see where does our SDK fit, where doesn’t it fit? And more importantly, how do we really align our technology so that we can power as many use cases as possible along the way? So, so two

15:29 thoughts, two thoughts here. One, I love this slide. I just little heart in the background. Yeah, yeah. Wow. It is it’s, it definitely makes me the most proud of all the slides that I have. And two so the runtime now, because we were talking about how the SDK was called the CLI SDK. So the runtime now is below that. So the CLI actually kind of uses that runtime, right. We are in the process. It from a history perspective, it all started in the cloud, right? Our GitHub copilot coding agent was the runtime in the cloud. And then we brought it down to machines

15:59 on the CLI. When we dropped it out of the SDK or I’m sorry, the runtime out of the CLI, we were in this transition state just because of all that history. And so right now we are replatforming the CLI on top of the SDK. Okay. Historically it’s kind of been side by side because we were intertwined a bit too much. The team did a fantastic job over the last 2 or 3 months separating, decoupling these two things. And so we do now have disparate parts. We’re restacking the CLI on top. So it’s another SDK client. Gotcha.

16:30 Just like the harness that’s going out in VS code Visual Studio JetBrains is an example, our GitHub copilot app and our GitHub CLI itself, they’ll all be unified on top of the SDK for the future. Totally makes sense. And it takes a little bit to it does, you know, unwind. There is not only are we making these big architectural changes, we’re also adding features. We’re also adapting to a market. And you know, there’s a lot to do. And there’s a lot of fun to be had at the same time. So one of the also the fantastic things

17:00 is when we ship the, the SDK, a lot of it was used locally. You know, people used it in a way very similar to our copilot CLI, which is a very local setup that users like to use in interactive mode very much, though a lot of that’s moving to more non-interactive use cases as well. But what was really fantastic about the journey as well is that we’ve now moved into the cloud very deliberately, and we’re actually moving into multi-tenant applications, which came with the need for some of these architectural

17:30 changes and more flexibility behind how you host and run the the runtime as well. All right. The other piece of it is even when you look across local and cloud and multi-tenanted setups, there’s still a lot of variety beyond that. So even when you’re in the cloud, sometimes it’s in like an enterprise VNet as an example, because we run Azure, there are things like sovereign clouds, there are air gapped from the internet. So this thing needs to run with and without GitHub, with our inference, with third party

18:01 inference, a whole host of standards that are brought in externally. Because GitHub is very much into the open source community and all the steering committees as well. So there’s a lot of mixture behind how we think about the runtime, which is why it must be extensible, it must be configurable, and it must be portable. Most importantly, to survive all of these requirements. Essentially, the last thing I’ll leave you with is I don’t want to waste any more time on slides. Slides are kind of boring, but they provide a nice overview. Yeah, no, the lays the context and

18:32 that’s great. And you hit on a couple important parts, like the whole portability. But we’ll talk about that once we get into that wonderful demo. Should we dive in? I want to first bring up a tool that I’ve been working on to help visualize all of this. Oh, yes. Because I think one of the challenging things about a runtime and an SDK is they’re really just libraries at the end of the day. And ultimately, it’s kind of hard to conceptualize what it is and what you’re actually building with. And so for most of this, I’m just going to use our shiny copilot app, which I love to

19:03 use as my daily driver for multi-agent orchestration. I have this copilot SDK visualization tool that I can just ask the copilot app to open up the app and canvas. And I think what’s great, I’m sure you use these all the time in the copilot app is we do have canvas abilities built right into it. And so while we’ll show it off, if there’s anything along the way that doesn’t make sense, let’s actually improve the app together too, because I like to use this as a feedback session

19:33 with perfect. Yeah, feel free in the chat, let us know. And yes, I drive my entire life, my work life off of this. Yes, my wife will not let me drive my personal life off of this yet. Not not yet. You know, I think she just needs a little bit more education on the tool. Yeah. Make her watch the recording of this. So this is spinning up. It’s actually looking at the project. It’s going to start the local app server and it should pop that open. But while that is finishing, do you want to take a little look at Tiny Tool Town?

20:05 Yeah. Let’s show it off. Yeah. Let’s check it out. Yes. So we got Tiny Tool Town. You want to talk a little bit about it? You brought it up last night. I brought it up. Right? Right. So what Tiny Tool Town is, was an idea from Hanselman Scott Hanselman. And it’s essentially this huge. As you can see, 604 apps, collection of really just vibe coded apps. So I shouldn’t say all vibe coded, but little extensible apps. And I think he even says it here. It’s really, they’re not super polished. They’re not meant for commercial use. They’re just

20:35 meant for somebody who really needed a problem solved or really just wanted to make something cool and then did it and submitted it here so other people can find it. And you can check out all the source code. They’re all open source. And it really is just kind of a labor of love. And Tiny Tool Town is the town where they all live, where you can go find them. So yeah, it’s just a collection of really cool apps that really just kind of serve one purpose and one purpose only. Yeah. And what I love is he equates it to the geocities of the current

21:06 time, which if you’re old enough to have played with geocities, I think this looks a little bit better. It does than the classic geocities. We weren’t quite good at web design or development back in the late 90s, early 2000. I would say, even though most of this is probably vibe coded, definitely more polished than the original flashing on here when I was when I was taking a look last night and I was just browsing through, I actually noticed Cassidy Williams has a little tiny tool right on the home page. And so I thought we would take a look at that tool

21:37 and see what it means to add an agent to it and go enhance it with the copilot SDK together. What do you think? Yeah, that’d be awesome. Yeah. Actually taking something that’s there, this is full feature, but yeah, let’s add an agent to it and see how the SDK actually fits into something you might even have today. So we don’t need a greenfield app to be using the SDK. Yeah. And I think that’s what’s really important is as you look at applications and you want to enhance them, you can stick an agent in just about any application in the modern world, as long as you think about it from the perspective that it is a little,

22:07 every app is like its own little mini operating system. You can give it tools, you can give it extensions. When you stick the agent in it, can it drive it? Can it activate certain functions that benefit you as the user? I find the answer more often than not is a yes, but ultimately it’s up to your creativity. Like what is the most value you want to get out of it? And really, what are things that you want to offload to an agent or call an agent into support as much as possible, right. But taking a look at this, this tool, let’s actually open up the website and see what Cassidy’s built.

22:37 Looks like a standard little markdown editor. Nothing too fancy a little there, which is a great tiny tool, as they say. I think what we’ll want to do is go ahead and just go to the repo, add it to the GitHub copilot app, and let’s just start hacking away. Yeah. And that’s one of the great things about GitHub copilot app. You just point it at a URL, add it, and now you’re working and our canvas is up. But we’re going to get to that in a second. I think we should put our agents to more work. And then let’s go dive in and talk a little bit

23:08 about the runtime. So inside of the GitHub copilot app, I’m just going to go ahead and add a repository. It brings in the fancy gist, which I can start a new session over here. And before we dive into all of the flexibility of the SDK, I think what’s really fun to show people is that you don’t really have to think too hard to add an agent to an app. We provide out of the box a very opinionated harness configuration, and while you can customize it all, it’s up to you how much you want to

23:39 customize it, which we’ll get into in detail. But I think what’s really amazing is you can just ask to add the SDK and you’ll see some, some easy stuff that happens along the way. So let’s just tell copilot to install the SDK, the copilot SDK, this is a markdown app. So I like to highlight text call into action agents. So I want to be able to highlight text and call an agent into action to support editing or providing

24:12 suggestions. Normally I don’t do this in apps, but because it’s great for demonstrating, let’s add a little side pain so we can actually see the chat. You know, I spent a lot of time trying to get rid of that chat in my modern experiences, but it is very helpful to show people what’s actually going on kind of from a geocities app. Exactly. Let’s add a side pane with the agent chats so I can see what’s going on and steer them. Steering just means as

24:42 they’re running, I can provide more feedback, I can engage with them, all of those things. And so this should be good enough. Let’s also double check our model settings so that it doesn’t take too long. Right? We were chatting about it a little bit this morning for context. You know, I like GPT six soul because I don’t want it to be too smart. Two state of the art. Otherwise it’s going to ponder probably try to do too good of a job. And then I want a medium thinking level because again, I want it to do a good job. But you know, we only have so much time for this thing to run on our side. So

25:13 we’ll go ahead and kick this off. The last thing I’ll note on this is we have built in tools for installing the SDK. So you will see it start executing a tool to install an SDK in an app. We do have helpers in there for you out of the box. And that’s what makes some of this easy to actually do with one single prompt. And we’re just going to leave it alone and go right into, we’re letting an agent add an agent to an app. Exactly. So I have this configuration tool that I’ve been working on over the last week or so, just off and

25:44 on as I work towards helping others really understand the runtime, the SDK, the harness itself, and with this transition from the CLI to the runtime itself, I think it’s really helpful to understand where in the stack we’re working on these things. And so when you look at this first overview page, you have an app, you have a platform, that’s what you own. You’re integrating the SDK into it. Just like we are into this markdown editor. The SDK is your primary interface. It’s

26:15 going to be your client. It is what’s going to instantiate your harness and runtime. It is what’s going to drive it. It’s what’s going to allow you to probe it, hook up to event handlers, etc. And then there’s the actual harness configuration. And most of the time is spent in the harness configuration for integrators. If they want a lot of control, if they want to generalize it in other ways, and if they want to do things that are a little bit different than what our default copilot harness does out of the box, but there is

26:45 the runtime engine. And so I think there’s a lot of questions around what is a runtime versus a harness. How do I think about what’s going on there? And the easiest way that I think about it is that it’s a little engine. It’s a little engine that could just like the tiny tools and tiny tool town. Matt, are you familiar with what goes on in the agent loop itself and, and the harness side and the runtime side? You know what, I kind of sorta am familiar and how it goes out and uses tools it uses. Once you get the tools, you can call out to the model,

27:16 or the model can answer what tools it needs. Let’s go over this. Okay? Because it’s, it’s, it’s cool how everything is how you have this set up really explains a lot of it. Wonderful. Yeah. Well, let’s jump into the view where we can trace an actual turn. And, and before we get too far into this, I’m going to mention that we will have the link. So you can go out and view this. We put it in so late I forgot to get it out to the team. But yeah, we will have a short link so everybody can go and view this later.

27:47 Yeah, we’ll pull it up on the the resources to make sure everyone has it. So the runtime at its core is a loop, as is commonly a node. And it’s a loop because it’s chewing through tokens, transferring context to models. Models are coming back with suggestions to run tools, provide answers, etc. but ultimately it is this loop that occurs over and over again as you engage with an agent. But what’s really important is around that loop, there’s a whole lot of concepts that enable that runtime to

28:19 gain power and flexibility, and also really understand how to be successful within the domain that it’s operating in. And so you’ll see these ten boxes connected to the runtime and various states. We’re going to start with the configure piece. Okay. Ultimately, when you bind all of these configuration settings and extensibility and abstraction points to the runtime, you are enhancing its ability to leverage context and access tools that can drive into a business. Very common

28:49 stuff. If you use the CLI, if you use the GitHub copilot app. But as you assemble the information through these areas, the really beautiful thing about the agent loop is you’re essentially wiring up tools that allow discovery. You’re wiring up tools that allow access to different model providers. You’re wiring up additional access to context and places that are hard to reach traditionally. And so when the agent loop first runs, it is assembling context. Think about the system prompts. Yep.

29:21 And identity of the agent. The agent doesn’t even know who it is. When it begins, right? You are giving it access to an array of catalog of M.c.p.s as an example. So remote MCP servers where they might be sitting over our business systems. When you look at GitHub, GitHub is a prime example where we have MCP servers, we have Clis. We have all these ways of accessing the context inside of our platform, and you can expand this far beyond the reaches of our platform itself. But once you get the context, the inference

29:53 inside of this runtime is not just limited to GitHub copilot inference itself. So whether you have a GitHub copilot subscription, a model provider outside such as Azure Foundry, OpenAI, anthropic, etc. it actually deals with engaging with those model providers and making sure that the complexity of their implementations is hidden away as much as possible. So one runtime can talk to n number. We even have support for things like local models in the current world. Awesome. So that’s like bringing your own,

30:24 well, not bringing your own key, but kind of well, yeah, we’ll dig into a lot of nuances. But you are right. You are right. Yes. Bring your own key is not really bring your own key all the time. Right? It is a little bit more flexible than that and creates a little bit more opportunity as well. But yeah, if we’re running local with something like a llama. Exactly, exactly. Yeah. Llama is very popular. I mean, when you look at connecting to a local model versus a hosted model with similar properties, not too different, but local really gives you again, that flexibility. Offline scenarios come into play from an

30:56 authorization standpoint. That’s another really critical component. When the agent wants to run tools, when it wants to do things that might alter an environment. For example, there’s whole systems bound into the runtime to make sure that the integrator can control these things. And then other systems can actually control these things as well. Right? So think about the difference between an integrator wanting to actually present a UI to a user saying, hey, the agent wants to perform this action. Do you approve? That’s a simple permissions handler. There’s

31:26 also things like policies that can get bound to the runtime. So if an IT department wants to manage an agent running on a machine, they can put hooks on that agent and they can actually govern and actually view what’s going on there. Block calls, prevent things from happening, things like that. So kind of like a data loss policy. Exactly. Because the goal of the agent is to operate over context, trigger tools, make changes, be productive. And so the permissions and hooks component is that gate before you actually get to that slightly

31:57 dangerous step of execution, you know, all, not all execution is good execution, but we do a great job as well as the model providers do. Balancing all of these needs to make sure models stay aligned and within safe guardrails for you to operate with confidence as well. And then ultimately, it all gets stored in the session and events of the actual agents, usually on a local disk. If you’re using a CLI or the GitHub copilot app, sometimes it’s in the cloud in our GitHub managed service, sometimes it’s your own service

32:29 as well. But the main point is once this, these records get pumped out to your session, you’re just going to go around the block again and again and again until the user has decided they’re done. Or ultimately the agent might be in autopilot and decide that it’s done itself. Right, which is wonderful. And one of the things that I always kind of have to get my, I guess my a mental block over is that we’re not just talking coding agents here. It’s all, I mean, you can live anywhere. And so I just kind of want to reiterate that

32:59 as we take a look through here, because I always kind of jump back to coding agents, you know, copilot itself, but we’re talking everything as we’ll see with fancy gist here. Yeah, absolutely. And it helps when I look at whether or not the agent needs an answer to a question. You know, this is why I like autopilots. So I can just let it let it go. Not available. This will only take a few minutes. And so why don’t we actually go ahead and talk about your statement, what the coding agent versus how to use the runtime with non coding agent configurations. Because

33:30 that’s really the first decision that you actually want to make with the copilot SDK in general. And there’s one important change that’s coming in 2.0. We’ve historically defaulted to a GitHub copilot coding agent harness setup. In 2.0. We’re going to default to something else that’s a little less biased, because how you apply the SDK is in such a diverse range of options. We’ve we’ve grown beyond just developer use cases being the core focus. So we’ll let that

34:00 run. We’ll flip back over to our little canvas over here. And let’s, let’s actually exit out of this and let’s go into the base profile. So inside of the copilot SDK, there’s a few options for you. And when I say few, there’s, there’s technically many, but, but there are a few that are really powerful. And the first, most powerful setting is the base profile. This is what’s going to define how configured your

34:31 harness is before you start making changes to it. Gotcha. And what I mean by that, is it really visible when we start doing a comparison. So I put it in a nice little compare tool so we can look at an empty state, a minimal state and a copilot state. Okay. Some of the things that are clear is one inside of the SDK. We do have modes and we’re actually probably going to introduce modes that are a little bit closer to these three. We technically only have two today. You either start empty or you

35:01 start with our harness configuration in there. And so empty is really useful if you want to just build like a chatbot as an example, you don’t need all the power of file systems. You don’t need all the power of a bunch of tool calling, though you might want to use a little bit of it. You really just want something that’s very pure operates on top of the model, but you have to actually start from the ground up. And so you’ll look down this column. All tools are explicits. You are replacing entirely the built in prompts.

35:32 You’re not appending, you’re getting rid of the GitHub copilot prompts in terms of the built in tools, because we do provide a whole set of them, you’ll notice zero. There are no built in tools that are added to it. Discovery, right? Like when you use a coding agent in a project or a folder, because we assume it’s a coding agent, we do check if it’s a repository. We do look for GitHub folders and things so we can load things out. But what’s really important is you can turn all of that off because for many scenarios, you want to

36:04 control how things get loaded, which the runtime allows you to do, but ultimately you don’t want it to auto discover and load artifacts that maybe you don’t have control over, right? And so you do have an ability to toggle these things. And as you’ll notice, all this stuff is turned on in the copilot configuration because this is how we love to work, right? And then when you get down to identity and policy and events, you’ll see the same thing. Biases towards local states. On drives, you will see bias

36:35 towards large output. It’s reading large coding files. It’s spilling over content, writing to disks, reading those things again off the disk. And so there’s a lot of flexibility to deal with large code bases, complex code bases, large execution jobs. And then as you get closer to the empty state, you lose some of that because it’s smaller package, can run freer, can run tighter, and not have all these dependencies hooked into it. Out of the box. Yeah. Cool. Any questions about the modes? No, no, that’s

37:05 really cool. I mean, and I guess. The there’s, there is a lot, a lot to this. And but one thing I want to point out about the website that we’re talking about here is that not only does it kind of give you a tutorial of what everything is and what the runtime is and everything that plugs into the runtime, but it helps you configure everything. Absolutely. So it’s the workbench and we’ll get into the configuration part of it too. But we’re just starting to yeah, we’re walking, we’re walking, we’re scratching, scratching the surface. And so

37:36 to your point, I did mention the, there are two modes built into it on the configuration item. You do have lower level access to configure that option. Because this is not just educational tool. It will allow you to configure harnesses. It’s kind of education. You can configure it and see what pops out. And that way it absolutely is because I added in a live plan which shows you, you know, what are your choices actually mean? Like, because as you toggle certain things inside of the harness configurator, you

38:06 are taking responsibility on, you are deferring responsibility or delegating responsibility to somebody else. And so it’s really nice to know as you make configuration, what are, what are you actually own? What are you supposed to do? And then where do you even liability sit? Right? Because in certain kinds of setups, agents are really powerful. What are all of your core considerations to make sure you’re doing the job well, safe and in a way where the agent is isolated, yet can be as productive as you want it to be. At the end of the day, that’s what I love about this website.

38:37 It gives you it shows you what you’re up against, and also not even what you’re up against, what you’re taking on. Yes, yes, but also how to do it. Yeah. And I think one of the largest meta lessons when it comes to the runtime SDK is that you ultimately can decide how much quality you’re accountable to. We spend a lot of time and a lot of energy building the copilot harness when it comes to how it’s configured, how it gets benchmarked, how it gets evaluated, how we work with third parties to make sure it’s capable. There is a lot of time

39:08 and energy and expertise put into our default configuration, but as you need more control, as you want more control, as you want to experiment, you can take that control, right? And I think that’s what’s the most important piece about what the runtime SDK really is. Before we go into configuration, let’s see if we got an agent out because, you know, hopefully this thing is stop asking me questions. And it’s given us a nice little agent we can work with. All right, let’s see if

39:40 it’s running. Oh, I should probably open the app in a canvas. Looks like we got a little bit more time. All right, we’ll keep it going. It does what you tell it to do. You know, these things sometimes will be two minutes. Sometimes will be seven minutes. We’ll see. But that’s okay. We have plenty of stuff to show and we can continue diving deep while we wait for that application to pop up. So let’s go back to the first decision that we need to make. What is the actual base

40:10 profile we’re going to use? Because we want to educate, we can educate on any of them. We’re just going to use the default copilot one. Okay. And so you’ll see in the live plan on the right that we are choosing the GitHub copilot profile. There’s a few other things over here, but we’re going to ignore them for now because we’re going to get into those details as well. Let’s go to the logical next step. All right. Which is the prompt. Yes. Every configuration has a system prompt. And we like to

40:40 make sure that you have full control over it at all times. When you look at the copilot setup, it’s going to default to the default because we expect you probably just want to use our built in system prompts and configurations, but we do allow you to append custom instructions so you can add on additional guidance. You can still replace the whole prompts, but typically you’re not going to start with GitHub copilot and then replace the whole prompt, right. Or you can do what a lot of our integrators

41:11 do, which is customize sections. Okay. This is something that came up through all of our deep engagements. Integrators didn’t really want to modify the whole thing. They just maybe wanted to switch the identity of it because we still report, hey, we’re GitHub copilot, those kinds of things. And so you can actually view inside of our runtime SDK what our prompts are. We don’t hide them away, we expose them. So you understand what’s going on, right? And like I mentioned on the identity front, you can choose how you want to deal

41:42 with the section itself. So you can replace an identity, append to it, remove it, etc. And inside of the tool, we’ll give you a nice view of what the prompt looks like. That way you can actually see it and then make your decisions as to what you want to do with it. And like I said, this first line is usually one of those lines that people want to change because not everyone wants to report themselves as an AI coding agent. It might want to be a financial reconciliation agent of some kind. And so people

42:13 usually focus on that. And then when you look at the system prompts, we do inject other things dynamically. So something this tool can’t fully show you is the values for these little fields that are marked up here. But it does tell you what it’s going to roughly inject when it’s running. And the reason why you can’t manifest them here is because depending on where you actually run the agent, it could come from the environment. For example, like what is the host operating system it’s

42:43 operating on as an example. And so we can bring in other contexts. But again, you can control how all that stuff is managed on your end. So if I go, if I’m an integrator and I go pick the copilot profile, would you recommend I go through and read all this first to make sure it’s cool or, or am I trusting? I mean, many times I just trust it to be honest. But I think it’s really up to your level of curiosity and where you find the SDK and the default profile isn’t fitting your needs, right? Okay. Most of the time you’ll end up

43:14 building evals. You’ll build some kind of way to establish a baseline in terms of quality. Depending on where that journey leads you, you’ll make decisions to satisfy and improve your baseline over time. Get a quick start and then kind of tune. Exactly, exactly. And so for this one, we’re just going to do a little append because otherwise, you know, a lot of the influence comes from the system prompts. And if you really want to feel some impact on the agent, just append something to the system prompt or modify it. In general,

43:46 something I like to do is, is adjust the tone. Being somebody from Los Angeles growing up on beaches, I like if I had my voice agents call me right now, they call me like they’re a surfer dude. And I think it’s actually just humorous when they call to ask me about licenses, they want to add to projects, but they do it in a very Southern Californian accent as well. And so, you know, use the tone of a Californian surfer with Californian slang. You know, I might live in the northwest,

44:17 but you can’t take the Southern California. I’m from Wisconsin. I would talk about cheese for a while, which is totally valid and acceptable as well. So we have a nice prompt there. Let’s leave it there for a second because we’re already on our way. I want to go check back in. All right. I have a feeling our other agent might be done by now, which means that we can actually pop it up. So let’s take a look. All right. It says that our UI is available. All right. So let’s go check it out.

44:50 Oh, it helps if I don’t bring in the ticks with it. Oh. Oh. Is it running on the wrong one. Let’s see. Been kind of a zap. Looks like we might have ourselves a good old fashioned bug here, Matt. Yeah. Let’s see what, see if it pops it open. All right, here we go. Yeah, it

45:22 might just want to close the reset. That one. All right, let’s reopen it. Different ports. Everyone can see how I triage and debug these things. Well, I mean, a lot of times when you are working with you can’t offload all of the understanding to the agents, right? It’s definitely you want to definitely get to like a black box state. And I try to do that all the time too. Like I’m not going to think anymore. But you still do have to think. You do rely back on your best

45:54 software development experience in order to fix stuff like this. Well, I’m doing slightly complicated things in that we are running, I think, two apps that wanted the same port. And so you just got to shift the port a little bit. Yeah. And how would you know that if you didn’t have that experience? It probed. That’s what happens when you choose a medium model instead of the high or the extra high where it starts probing what’s I see, Patrick, I’m trying to give you credit for being super smart here. And you’re saying, no, no, no. Oh, it’s, you know, all agents up and down. All right. So I can select some texts. I do see

46:25 some buttons, you know, could have done a little bit of a better job, but that’s fine. Let’s talk to an agent. Can you make this better and more formal? You know, I love using words like better, partially because I don’t know what the agent’s going to do. Better is a very loosely defined term. And so I find that asking better is just fun, because you’ll figure out what better means to a model as you try out different models and patterns too. So let’s see if this thing goes, oh, it’s still starting

46:56 even though it’s already started. All right, it’s all right there. So I’ve got my edit. We have a little formatting. I think that we want to work on. String doesn’t work. You know, I don’t like this bug. We’re going to keep it going. One of the things that is great about the copilot app is, you know, you can just pick and polish. This error is popping up. Let’s go ahead and polish these buttons too. Just

47:26 because my OCD does not like poorly formatted buttons, fix the formatting on these buttons too. All right, we’ll go and be a little bit more productive while this thing is on its way. Yeah. So the nice thing is it did stick the SDK in. I’m sure we can actually run it once we fix these things. And then we’ll go ahead and flip back on over to our app, keep the customization journey going. All right. We’re setting up the other one. All right. So we talked a little bit about prompts. What other areas are

48:01 on your mind related to prompts. Any any questions about capabilities? What kind of things are on your mind related to prompts that? Well, after that, after we have the basic prompt, right? I want to kind of get into my domain of what I’m building. And so again, I go back to my coding agent experience because that’s just where my mind always goes. And so I have my custom instructions, right. I’m going to also go back to skills and like MCP servers. I want to give it that extra knowledge

48:31 about my domain. Yes, absolutely. Are we on that page? Is that what we’re going to be doing? Oh, I mean, we can dive right into tools. Absolutely. To what you’re saying, tools are one of the most important things, if not the most important thing. Because ultimately the catalog you present an agent, you know, will bias it towards certain actions, depending on what tools it’s use are available and the prompts that you give it for the job that you’re trying to do. And so when you look at what’s built in to the runtime and SDK, we do have

49:03 built in tools. You can bring custom tools, which are actually tools that you can bind through code. So if you’re bringing this in an app, you can write tools and the same language, hook them to the runtime and they will execute as part of the agent’s lifecycle. And then you have, of course, other tools like MCP servers, which can be local, it can be remote as well. And then you can combine all of these into a tool set that is then presented to the agents, and they can leverage as they see fit based on your system instructions and your prompt.

49:33 So there is potentially a prompt work you might want to do. Again, it’s a coding agent. It thinks it’s there to code. And then you have the entry prompt if you want it to really do other jobs and bias them towards it. You might want to adjust the instructions, but you usually just putting the right tools in there and it picks it up a place to start. Yeah, absolutely. And even flipping back to the application that we’re currently on. It probably put tools in. We can’t see them

50:05 quite yet, but it probably put tools in to allow it to modify the document, understand how to work within this context. And so let’s actually just go ahead and see if it actually did it correctly. Let’s go. Let me clear these chats. Do a little refresh. Perfect. Hi there. So quick, easy test. Let’s see if we can make this more formal. Can you make this more formal.

50:37 Wants me to log in. Let’s see if I can log in with it. So we’re logging into GitHub here. Is that. Yeah, probably. We picked the GitHub profile. So gotcha. The auth mechanism should offload the rabbit holes go deep. I don’t know if I can auth within the app within the app you know. And so there could potentially be some little thing there. But let’s see. Yeah, I want to do this

51:07 with the built in GitHub auth. Don’t require a login. Use the CLI on my machine. So one of the advantages for local we don’t actually need to auth. I have the CLI. I have the runtime running on my machine. I’m already logged into GitHub. We can just adjust its configuration so that it’s just using the built in which should get around this need to log in. It’s just going to look at my local setup, use my local logged in account and then be

51:37 able to interact. I think it took a shortcut and was trying to get around the need to build back in service, but it should build me this back in service to make it work real quick. One of the reasons why it wants to log in is because it’s actually using gists in the back end. Exactly. So that’s why it wants it. It wants to use gists. It’s not it’s not doing anything. Well, there’s gist and I have a feeling it’s trying to ride the inference over it as well in one. So it was actually trying to do the proper thing. But I wanted to do not the proper thing. I just want a little local tool for our

52:08 demonstration, because we don’t really need all the complexity that a proper login flow means. In this scenario, half the time that I’m using the SDK, I am just crafting tools and things I need and instances. And you know, I go for the bare minimum setup. I don’t generally want to build whole login experiences, even though they’re very easy and quick to build itself. You do need separate services and other things to make it all work correctly with the runtime as it runs. And so, you know, we’ll just update it so that it can actually find the CLI path, use that locally. And then that should resolve everything that we need. Because when you get

52:39 to a hosting environment, you can run it using local CLI as your server. You can have the SDK deploy its own runtime and actually use it, or you can embed it directly in an app, which we’ll touch on a bit more in the in the future here. All right. Yep. So it’s probed. It’s found the CLI and it’s validated that it can in fact answer things through it. Let’s go on back to the canvas shall we. All right. All right. So we

53:11 talked a little bit about tools, but what are tools? There are a lot of tools that are built in by default. We have ones that are available to the agent right away. And then there’s what are called conditional tools, which show up roughly when an agent needs it for a certain context. But when you look at this tool, we will actually give you full control over these tools as well. So basic copilot configuration, you’re probably not modifying tools. You’re going to add

53:42 tools, right? Okay. But there are definitely reasons why you want to remove tools or go one step more advanced and override them itself. So you’re using our system prompts, you’re using all the same logic, but when it calls to create a file, to edit a file, to read a file, you can replace that implementation entirely. Okay. And so it just touches on another component to extensibility. Not only can you modify the whole system, prompt modify sections have that granularity, but even when you

54:13 get into the tool catalog itself, one of my favorite things to do is override the ask user tool with a voice ask user tool that calls me and asks instead, because it is actually much more entertaining, I find, and then slightly more stochastic in terms of where it can go. And so I love to actually build a tool where it calls me on voice, does the thing it wants to, and then ends with a joke just for no reason other than you can in the modern era. We might as well have a little fun, but

54:43 from a runtime defaults. You will see what the tool does. Like I mentioned, you can overwrite it so you can actually define a handler for it that will be triggered whenever whenever the model wants to call the ask user tool. And then there’s custom schemas. You can give it, you know, when the tool is called, what are the input parameters to it sure doesn’t have to map to ours. You can actually redefine what parameters the model thinks it should be filling out and making it available. Or you can just remove the tool. And so as

55:15 we’re making even these changes, if I flip back over here, you can start seeing deltas, right? We’ve tweaked the prompt. We’ve now tweaked some of the built in tools because I just marked one of them as removed. We’re definitely like the user tool at the end of the day. So we’re going to want to keep it right, right. So if I just go to over there, let’s just go ahead and keep that tool. But if we continue to take a look at the everyday tools, you’ll see a lot of things that I think you would expect. You know, it’s ability to run different shells,

55:46 it’s ability to read files, modify, like I was saying, to search over things in the file system, seek a lot of the tools that are built in, again, are going to be for coding scenarios. And that has to do with hunting through your system mining, dealing with lots of diversity in the types of files that you’re currently trying to understand and extract information from and leverage as well. If you get to the more advanced capabilities, you can go a little bit deeper. We have even more biased tools in how they work, which again,

56:16 you can decide, you know, should I include these? Should I leave these out, you know, LSP tools for language server protocol? Probably not going to be used a lot outside of coding scenarios, those types of things I would usually opt to take out if I’m not going to actually leverage it for coding scenarios as an example. But again, total flexibility. Yeah, you can override, you can remove, you can add. And then when you get to custom tools, you do have an ability to bring in custom tool names, custom

56:47 tool descriptions. You can do all of this in code as well. And so you do have the flexibility to modify the entire catalog, not just deal with the catalog we provide you, which again, is probably the number one thing that people do. They’re always finding new tools to give it some kind of access or expertise in a different area as well. So do you have any tools that you won’t let people take out or like? Is the minimal, the empty configuration? Oh, it’s gone. Yeah. It’s totally empty. Yeah. We, we deal with so, you know,

57:18 take for example, some of our collaborations on the multi-tenanted side. Those agents, you know, can just be chat interfaces plugged into other platforms as an example. So there is no file system. So you’re going to strip every tool that thinks it needs to actually interact with the real file system, and you’re going to bias it towards tools that are designed to just integrate with services instead. Okay. And so like those setups, you’re trying to shrink it into like a really small package. You want it to have really fast startup times. You want it to

57:49 run in really constrained environments, and you also want to virtualize the agent so you can run a thousand of them for different users without worrying about agent sessions. Cross-pollinating across users as an example, right? There’s real security concerns in those environments, but you also have the flexibility to share things in weird ways, right? Like how you configure things out across users and stuff is interesting. And it’s not even constrained necessarily to one user because the runtime doesn’t really have a concept of identity, it’s

58:20 provided to it. Which means is this a single user thing? Is this multi-user? Is this an agent user? You know, they all have to be supported by the same runtime. And so the host and integrator provides the right concepts for what the identity is in, in certain situations. And then we leverage that across the runtime in a bunch of different ways. Okay. Yeah. And then of course, there’s the good old MCP servers, which are wonderful for adding remote access. We have a great suite

58:51 of tools over here at GitHub. We have a great suite inside of our other organizations like Microsoft with our IQ tools. As an example, I love MCP in the modern era because hook it up to everything. Yep. You know, teams, Slack, GitHub, email, calendars, everything. And I, you know, I think everyone has their own personal assistant now in many ways. Yeah, I think so. Sucks in the context and makes it available and allows me to deal with all of the noise and extract signal as best I can. Well, we just move quick. Have you named your personal assistant? I just call

59:23 it my app, just whim. I’ve actually shown off in one of the live streams before, because I look at it from the perspective that anything I want should just be a whim. It should be very low effort. As one of my my good colleagues, Jayce talks about low ceremony at all times. You should be able to just, you know, cast an intention and get what you need and get what you want at any point. Yeah, mine has started calling itself the duck. Yeah, I expect at some point maybe I’ll personify it a little bit better with the name. But if you actually go into my tool,

59:54 I’m just, you know, mentioning a few different variants of what I like to do. I’m pretty bored on that front. Like my, my editor is just an editor. It’s not like anything crazy I want to be. I didn’t name it. It named itself Duck Emoji. Yeah, that’s the duck. I mean, my, my, my claw at home definitely has a name. You know, I run that on a Mac mini and I call it Pac mini because, you know, Pac used to be my, my nickname when I was younger. And so, you know, that stuff has, you know, I message it, you know, with an actual name,

1:00:25 but for, for stuff like this, I’m usually trying to aim more on the productivity side. Gotcha. And so I’m slightly more boring, though I do spend more time creatively building applications and other things. Okay, well, back to the talk here. Yes, let’s go check on our text editing agent because I am determined to get this basic one going. You know, should have probably tried this one out first. All right, all right. Make this more formal, easy test. Easy test. Let’s see.

1:00:58 Applies. I don’t know if that was more formal or did it change it to hi there. Let’s go. More formal. Hello. All right, all right, all right. Okay. We’re in business. Yes. Yeah. Okay. Hello is, I would say, the start of a love letter or something of some kind, but that’s okay. So we have an agent in here. It’s just a copilot agent. So let’s actually just play around with it while we’re having a little fun. Let’s see. So we should just be able to. What’s a good

1:01:30 topic? You know, we’re always talking about the weather in the northwest. Let’s see if we can go be somewhat useful edits. Let’s see. Oh, it hooks up a workflow right away on the edit. I, you know, I want to be a little bit more specific on these things. Write me a quick blurb on the weather in Pacific Northwest. All right, let’s see if this needs to go search the internet or just says gray and

1:02:00 cloudy. Like any reasonable model, right? Yep. Pretty much. I don’t even think it went to go look at that. Yeah. You know, it’s pretty standard day in and day out. So it’s in the training data for sure. Yeah. No cool temperatures, cloudy skies on and off rain, occasional sun. Yeah. That sounds like it’s been trained pretty well on the Pacific Northwest. But let’s go ahead and switch back over to the the actual app and continue configuring this thing. All right, let’s get back to this.

1:02:31 So we talked a little bit about profile. We talked about the prompts. We’ve talked about the tools. I think what’s really valuable is to get into the context and Pax, talk about some of the other ways we can configure the agent. And then let’s get down to how do you host the thing? How do you run the thing? How do you actually improve the harness itself? Going to context and Pax, this is where there’s a lot of knobs and levers related to discovery. When you think about our GitHub copilot agent and you run it in

1:03:02 a repository, it picks up a lot of stuff. It’s seeking, it’s hunting, it’s trying to figure out how can it activate all of this configuration and what is the user’s real setup that it should account for. And so you have simple concepts like working folders, which is usually the current working directory of wherever you’re working. So you can leave it blank and optional. There’s configuration discovery. So for example, do you want to pick up the GitHub copilot configuration from the settings folders that we put it in?

1:03:33 Because maybe you want to supply your own instead. And, and you’ll want to supply your own when you are doing like per user type things on a machine. So each user might have a different configuration. So you want to load it dynamically. But if you’re on your machine doing things your way and you just want it to pick up things that you use with the GitHub copilot CLI or GitHub copilot app, we can just turn on configuration discovery. It’ll go find those things. There’s skills which are really popular. People put skills into their

1:04:04 home folder. They put it into their repo. You can turn those pieces on as well. Git operations right. Not everything’s a repo, but if we’re dealing with code everything’s repo generally. And so we can turn those on too. And then there’s the opportunity to also define additional directories. And the reason why this is important is when you run an agent in a hosted environment, as an example, you might want to mount a folder into a container or mount a directory, or have

1:04:35 many directories of skills plugins. This allows you to actually specify one or more paths where you can actually bring those in. In addition, so as you spin up the runtime, it’s not just going to search and its natural location, it’s going to search in custom locations that you define as well. One of the exciting things that’s coming, it’s actually in our runtime where we’re about to expose it through the SDK is our skill provider. What this means is now, instead of it being based on file systems, you can load these skills from servers

1:05:06 instead. Oh, cool. In other abstractions. And so this comes into play when you want to host the catalog and then have it dynamically bring in that catalog from somewhere else, it’s something coming very soon. We’ll probably be out, I think, in our next release, if I’m honest, but it just enables one more extensibility point beyond local file systems as well, which is great. Touching on some of those multi-tenant capabilities that I talked about, we’re going to leave a mostly standard setup there as well, and then we’re going to jump to another area that uses

1:05:36 our should be mostly familiar now, which are custom agents inside of the runtime. You can define custom agents, bind those through the SDK, make them available to the model. So we have a reviewer similar in our app. We can have an editor. And really tt’s where you can give it a nice description and a prompt. I prefer very proper English. You know, as an example, you can define a model preference today. Model

1:06:07 preferences are truly just model preferences. So if you set a model preference on a custom agent that you bind into the runtime, it’ll look for it and try to use it and then fall back in the future. We are looking at putting in things like classes or families of models. That way it can fall back to a series of things. So if you have a custom agent that’s like needs a state of the art model designed to process really big jobs, complex jobs, you could put a few models in there across different families. That way, if there’s a problem on a

1:06:37 machine, you might run an agent on a machine. That user doesn’t have access to that model. There’s some optionality there, but we’re just exploring that at this stage personally. And then most importantly, you can change the root agent. So even though you’re defining custom agents, you’re saying when I start a session, this is actually the agent I want to be dropped into, okay, which allows you to control out of the box. What is the agent that’s actually going to interact with? Instead of just relying on your system prompts, for example? That’s actually cool. Yeah. Because that’s

1:07:08 always your kind of casting things to the wind and hoping for the best. You know, to a certain degree, you’re still kind of doing it. But yeah, I mean, ultimately it is always about the reps, right? You have to put in cycles with agents and configurations to understand the experience of them. And given how generalized these things are for many use cases, one of the battles is just figuring out where the boundary is and the limits are on your configuration itself. And then one of the other areas that you can do is you can choose to hide tools from the agents, like exclude them from

1:07:38 the view. That way you can remove capabilities if you want an agent to only be responsible for orchestrating other agents, it might ask users questions. It might do some basic things, but you know, you want to farm out all the reads, all of the workloads that interact with file systems, things like that. And so it gives you that flexibility to take certain capabilities and give them to other agents. So it comes down in our configuration file. We still have it defined There, but we’re just saying, hey, agent A, you can’t have access to. Exactly. And so if I flip

1:08:09 through because it looks like our page refresh, put on the copilot profile, come back here, you’ll see a whole host of tools that are available by default. You are trying to take these away or give them to the depending on what your goals are, which enables it to be more focused. Ideally, in a perfect world, there would be almost a perfect match between the tools that are available and what an agent is exactly trying to do, right. In the real world, it’s a little bit squishier than that on average. And so you try to get it within

1:08:40 range and mostly just use it from the perspective that you want to take away capabilities that it definitely should not leverage in its cycle. Yeah. When you get to the actual mechanics of how a lot of this works, one of the, the other big decisions is inference. You know, there’s a whole host of things that come with choosing the right inference provider. And one of the first things that we started working on right after we shipped technical preview of the SDK was bring your own inference,

1:09:11 because ultimately, when you think of deployment environments, there’s some that can’t talk to GitHub. So like, what do you do right when an agent literally can’t talk to GitHub? And so we have two modes behind inference. We have the built in GitHub copilot account, which we actually just used in the demo for that markdown application. That allows you to not worry about a lot of things. You don’t worry about accounts, you don’t worry about billing, you don’t worry

1:09:42 about inference. You essentially get all of that out of the box and you get it on top of our copilot API, which serves developers everywhere, right? When you think about how it can be used in the SDK, you can go much broader, right? We’re talking about personal assistants. We’re talking about multi-tenant scenarios. You can actually set up a GitHub copilot app, let other users log in with their accounts and then use their inference directly. So it’s fantastic for internal tools as an example.

1:10:12 Yes. And that was the most popular original use case because everyone started with Clis. They built little things on their machines and they’re like, how do I help my PM team or my engineering team? And so then they built tools and let people in their organization log in with their subscriptions into the tool, and now they’re using their inference instead, right? Which means everyone gets their rate limits as they’ve had them. Everyone uses their own inference, even though they’re all under GitHub

1:10:43 copilot accounts as an example. And it’s turnkey, you can light up an application in minutes, in effect by just adding a basic login flow to it and making sure you can get tokens the right way. That’s cool. Yeah, yeah, that’s really neat. One of the things that I will add before I go into the bring your own inference, because I think that’s really interesting. And a big use case is we do support a few things. So one is you can let a developer use their signed in account on a machine. We do have per session

1:11:13 token callbacks, which mean as you create sessions, you can actually force it to say, hey, give me a token, which can be a new user if you want it to be for certain types of setups. And then inside of GitHub, we actually started giving to some internal teams. Soon enough we’ll be sharing it publicly. GitHub app service identities as well. So now you can get higher rate limit accounts. Yeah, everyone’s wanting to build AI factories. Yeah. Everyone wants to set up automation and GitHub actions pipelines as an example. And so

1:11:44 our new service identities, which we just started sharing with some customers that you can actually see some docs in our copilot SDK docs even today will allow end users and developers to create service accounts that in essence can allow them to burst much higher, right? So you’ll see many more workloads, even though it’s GitHub copilot custom with the SDK and using these higher rate limit accounts that are available to them as well. And that’s like one account. Then for one account, for one app, many users. Exactly, exactly.

1:12:15 Or just, you know, energetic user, right? Sure. Right. You want to like queue a job up, let the factory run. You want to just isolate that around a service account. Yeah, but maximum flexibility, a lot of it just starts with teams are trying to run their own evals and benchmarks and actions. And, you know, those things can be pretty heavy or they take a long time because you’re trying to run them under a single user account, dealing with rate limits as an example. Yeah. And then you always have an option to define default models, default reasoning efforts,

1:12:45 context tiers, etc. meaning that when you activate the session, you come in with certain setup on that piece as well. Bring your own inference opens up a lot of doors though, and it’s one of those things that is really enabled us to go beyond just the GitHub copilot ecosystem and really support our broad first party partners and third party partners as well. The reason for that is we need ways to access other inference. You know, when you need certain data residency or

1:13:16 data submission guarantees, when you need guaranteed throughput at a real platform level. Yeah. Our GitHub copilot product is designed for developers and it’s designed for mostly single users, though that’s growing as we just talked about. So if you really need scale, you’re probably in a bring your own inference type of setup, right? And so we do support things like Azure inference, OpenAI anthropic standards as well. You can define your provider wire API.

1:13:47 So between responses and chat completions, you can bind credentials different ways into the runtime to make sure that environment can provide them. Or again, there’s callbacks where you can inject them. And then when it comes down to the model behavior, you can provide your own model lists. You can allow users to select them, you can wire in all of that so that you can make it available to whatever client or integration that you’re actually trying to solve. And that’s really one of the more, I don’t want to say the it is a powerful thing. There’s so many powerful things

1:14:18 here, but having that choice of what you want to do or what you can do with it, I mean, that’s a big deal. Just like data sovereignty to be cut off because there is a lot of companies who just can’t. Yeah. And be able to still use this. That’s great. Yeah. And I mean, the pragmatic realities is, you know, a lot of companies are multi-cloud. A lot of organizations want to experiment. Yeah. We’ve been talking about third party inference as just managed services in the cloud. But people want to run local models. We have work on our side to

1:14:49 support local models, for example. And so when you look at all the variants behind, how do you get inference plugged into the runtime, making sure that we provide a great out of the box set of options is really important. And then also making sure they’re extensible too is really important. And so as we have teams across GitHub and Microsoft, for example, trying to ship clients as fast as models are coming out the door, there’s enough points built in and mechanisms that allow you to own your ship

1:15:20 schedule and your quality ultimately, which is the most important thing because as you take a dependency on the SDK, we don’t want to be like the blocker for delivery timelines, right? Right. As an example, you can imagine what happens if you hold up some of these trains and some of these things are trains that just move on a schedule. And so it’s really important. We built a SDK in a runtime that was flexible enough to support all of this. Well, again, getting out of the way of the integrator as much as possible. The other area

1:15:50 that’s really key are the policies and how we manage state itself. So when you dig into policies themselves within the SDK, this is actually kind of funny. When we ship the technical preview, when we ship the technical preview, it was the CLI SDK and everyone assumed, because it’s the CLI SDK, everything about the SDK was like a CLI. Kind of fair. Yeah. Ultimately, but one of

1:16:21 the things that we did that at the time felt like a good choice for a library in SDK ended up being one of those choices that we reverted probably within a month or two, which is if you activated the SDK and you used it, we did not force you to add a permission handler, meaning the agent loop when it was running was in a YOLO mode all the time, because ultimately it’s up to the integrators to define what should happen and where and where it’s put in terms of

1:16:53 containment. You know, again, that’s a host implementation detail. But what we did have to do was make it a little bit more explicit. So. And so what you have are pre tool policy and post tool result hooks, which allow you to inspect payloads, block tools from being called, and then also transform payloads as they come back from inference providers, for example, or tool providers. We now make you set whether or not you’re going to give us the handlers or you’re going to explicitly YOLO this runtime.

1:17:24 Yeah. And so if you do explicit allow all, you don’t have to give it the code handlers for it. It’s just going to run the loop in a full autopilot. For most implementations, you will probably bake in a pre and post tool only because you can still YOLO it, but you can make decisions as things are happening. Yeah, people are building learning modes, permissive modes, things like that where you’re kind of looking at behaviors as things are executing and then making decisions as to what is what risk levels really mean. And

1:17:55 then like what actions should be categorized at what risk levels, things like that as well. And so there is flexibility there, but we’ve switched it already. It was one of our earlier breaking changes to say, hey, we should probably not YOLO this thing by default, because while there was a lot of integrators that understood it, there was still the hobbyists coming in building personal assistants that just really pressure tested. It could be a surprise. Yes. Yes, absolutely. Absolutely. Then there’s a whole hook based system in it as well. So you can pick up hooks off of the

1:18:26 file system, or you can bind hooks directly in the runtime. We do have a range of them. And the nice thing about the hooks built into the runtime is, you know, some of them are just about observation. Like, let me see what’s happening. Some of them allow you to block certain actions, right? So you can use them for governance scenarios as an example. And other ones let you inject certain contexts back into the stream. And so we’re always looking for opportunities to make it more extensible in this area, expose more lifecycle hooks and stops

1:18:58 along the journey. But for the most part, they’re really powerful, very nicely extensible, generalized to support enterprise, the hobbyist scenarios. I use them in fun ways, and then we also use them in serious ways. It’s kind of the meta points, but total control again, over what gets bound and how it gets executed. One of the interesting things that has continued to be pushed on is our session state lifetime. Where is the session actually stored? Yeah, yeah. Where is it stored? I don’t know how you

1:19:29 feel, but like, have you felt a little constrained to your machine on some of these agent workloads? You know what? Yes. Everything is actually just essentially files. It is on my machine. And so we have we’ll touch on the architectures in a few minutes here. But ultimately everything was always file system based up until I think it was 2 or 3 months ago. We started to add it in and we continue to pressure test the support for it. But we do have host file system adapter built in, so you

1:20:00 can start abstracting away the session from the file system itself. We’re using it with a few partners that are launching soon, but it allows you to get away from everything. Must be on the disk, has to write to a disk, has to be local. I need a full container. I need to have a way to actually write this out securely and keep it away from other users. A whole host of other problems are eliminated. And then at GitHub, we also have our own managed servers that we’re integrating things into to make sure Session State is portable and accessible from other places as

1:20:30 well. You can turn on and off a few of our additional toggles. We talked a little bit about like large output handling agents don’t do generally well with lots of super large files. And so even simple things like pasting a big blob of text, we might write it out to a file and then parse chunks of it in different ways. That’s cool. I mean, that’s just handled absolutely, absolutely. But again, if you don’t have a file system, where do you write it to? How is all this stuff happening? All of those things

1:21:00 you need control over? And then there are a few settings that I definitely do recommend keeping on, like streaming and the event observer. Otherwise, you’re just going to get like final turns on things instead of the actual streaming tokens, for example. And then the whole runtime is an event sourcing architecture. So it is pumping events the whole time and you can wire yourself into them, listen to them, decide what to do with them. Right? We’ve really increased the fidelity of the information that’s being pumped out, but we’re always looking for feedback and opportunities to create more

1:21:32 opportunities to extend and interesting ways and stuff like that. So cool. Yeah. Once you have all this stuff configured, put a little build and run or a little builds and run page in here. Nice. That talks about a few other key decisions. Okay, so six languages. What’s the language? It’s usually an important decision. Yeah. But is it always. That’s a whole other philosophical conversation or language always going to matter? Probably not.

1:22:02 I was I will actually touch on why I don’t even think I need this anymore in this tool, because really with, with the power of natural language processing in the current world, capturing your intents allow you to carry them into any language. Yeah. And I think as we move to a world where we’re trying to really capture clear intent so we can understand what the output is supposed to mean and what it’s how it’s supposed to operate. I look at this as kind of going away

1:22:33 because it’s more of an implementation detail, like when you install the SDK. I didn’t tell it to go get the TypeScript one. When we did that little tiny tool, I just told it, it’s going to look at the language, it’s going to try to make decisions. And so it’s kind of like when if you think about funny things like blood types, when I was younger, everyone knew their blood type. And nowadays it’s like when you need to know they’re gonna, they’re gonna check it. Yeah. And that’s how agents are. It’s like everything is pretty much on demand when you need it. It’s best to assess the situation and then apply it to

1:23:05 that environment or situation. But for all intents and purposes, we’re going to leave a TypeScript in there. One of the great callouts for 2.0 that’s already available in a prerelease is with the rust rewrite. We’re able to do native in-process runtimes in proc stuff for the first time. So if you look at these first two options, the SDK can spin out new instances on its own if you want to just configure it that way. So you can bundle the CLI in. We actually updated the CLI license so that you could bundle it and not have any concerns for real commercial

1:23:36 applications. Or you can spin up an existing runtime on a machine, which is how we actually fix the tiny tool. We’re like, hey, just use the thing that’s already logged in, or you can switch now to native in-process, meaning you don’t need to spin out an external process at all for any of this. You can actually run it directly in an application, which is one of those things that unlocks a whole new range of applications and ways to use the runtime and SDK itself as well. For this one, I just like to, I’ll just use a managed child process just so we can

1:24:07 look at what a configuration means. And then I think this thing did reset on me. So I’m going to, because, you know, I don’t keep state across page refreshes yet. It’s not that advanced of a tool. We got to bring. California slang. And with a positive attitude. Yeah, yeah. We don’t want we don’t want we don’t like, we don’t like negative attitudes, you know, and so when you come back here, I have this little plan on the right, you’ll see application process host SDK,

1:24:37 SDK owns the child. So it’s going to try to configure for that. We didn’t customize too much in it because we just really want a basic setup. And then it actually has an open AI config set on it as well. Okay. At a glance, you know, I produce a few metrics on what’s been changed and then ultimately really internalizing what these decisions are, right? So runtime default selection, you can click on each one of these to really understand what, what are you supposed to take accountability for. And then there’s a nice handy export

1:25:08 plan in here. So it gives you code in the language of choice. So you’ll see where it’s actually create session. All the options that we selected. There’s copilot CLI instructions. Because if you just want to paste these things in, let it rip. You can or you know, the tool technically has a little import export. It vibed itself in there. I didn’t ask for it, but you know, hey, it’s useful because we want to speed it up a little bit. Let’s take a little look at grabbing the instruction that we want.

1:25:40 Let’s just grab this one, close that out. Let’s go flip back to our tiny tool and see if we can get this thing to meet our communication standards. Well, that’s running. Let’s talk a little bit about hosting. All right. Because that’s the last important topic. And then hopefully this thing is gonna come back with a real positive attitude and California slang. I added in a really important section, I think because

1:26:11 hosting models are really diverse, and I think sometimes it can be hard to conceptualize how you’re actually hosting this in different scenarios. And so when you think of a personal assistant like we’ve been talking about, you’re probably have a little app on your machine. Yeah. Which if you’re curious, you can actually click on all these and it will, it will break down these things. So we won’t go through it at this time. The runtime runs in its own process traditionally, unless you’re doing in-process configuration, has access to all your local

1:26:43 tools, and GitHub provides your inference, stores your sessions in the cloud. Really simple, basic setup. It’s probably what most people start with first, and then you expand to more complexity from their local containers is usually the next step. And that’s where you get a little bit more complex in terms of your app communicating over TCP to the runtime that’s running in a safe container, which unlocks an ability to execute all these fun workloads

1:27:13 without worrying about your local machine as much. And it allows you to start configuring things in a slightly more production state, if that makes sense. So, for example, the container has an egress allow list. What is the network? It can actually what can it talk to over the network as an example, things are getting persisted outside of the container. So as you spin that container back up, as you rehydrate it as an example, how does the information get back into it and become accessible to it in a durable way? When

1:27:44 you think about the secrets themselves, you can start keeping secrets away from agents so that they’re not actually engaging with the host operating system in a way where they’re accessing all kinds of things. You can control what information it has access to very deliberately, and then this gets scaled up in a real dramatic way to a real production implementation where you have things like gateways, you have services, you have policy approval engines, you have credential brokers, session stores, you have telemetry pipelines. Tried to

1:28:16 present a really good visual of the types of architectures that teams are deploying. This is in the real world, you notice that you have runtime pools instead. You’re not worried about a single instance, right? You want to spin up 1000 of them. You can really scale this up from small versions, like on your machine or these really big production grade versions of our runtime running in architectures that support, you know, millions of users daily for the most part as well. So really powerful, really, really amazing kind of topologies can

1:28:46 be wrapped around the runtime. But again, the number one thing is you take accountability for the complexity and you decide how you want to operate in those environments itself. Yeah. If we flip back, I think it should have applied. And then we can bring up some reference links, let people play with it. Yeah for sure. All right. Perfect. Let’s go back to the text. It looks like all right. It says it’s got a positive attitude. Let’s see how positive this is. You know how is weather. Here you go. This

1:29:17 should be a yeah. You know it’s always a good always a good question. Because you know I think there’s two sentiments that would be expressed generally only one of two ways. You know, it’s going to tell me. Did it nuts. Oh I bet let’s see. Let’s see if we do a reload. You know, it’s more fun when it talks with. All right. Oh. Oh I

1:29:53 detect some Californian in there a little bit. Totally. Yeah. Totally is definitely one of those things. What else can I add? Let’s see if we get one more out of it. I feel like that’s very mild for. Yeah. Yeah. It’s well it’s early for sure. I mean, I mean it’s so close to there. It really wants to really wants to spit it out, but we’ll call it a success. There’s, there’s more fun in games that can be played with it for sure. And ultimately,

1:30:23 again, you drive the experience and the more you can get away from chat into embedding experience, the more interesting I think the experiences are. Overall, as we wrap up, flash a few links on the screen. Perfect. Do you want to walk through the quick list here of the tools and links? Maybe I can zoom in for you. Yeah, Zoom in for me a little bit. Yeah. Let’s go. Cool. All right. The copilot SDK repo, if I’m going to be right, that is the repo. That’s where all the building is happening right now. So yeah, actually what I love about this,

1:30:54 you can actually see everything that’s going on, all from the PRS that are getting put in. The Readme is always kept up to date. There’s a changelog out there. There is. And if we were talking about earlier this week, is you can actually point your agents at that changelog. Oh, absolutely. Yeah. It’s pseudo meant for humans. Pseudo meant for an agent. Yeah. So that as you after you install the SDK and any time you want to do updates, you can just build like a little workflow on GitHub, say track the changelog MD in this repo for me, open PRS with updates as they come and it’ll look at improvements,

1:31:25 breaking changes, make suggestions, and we try to make the upgrade path as seamless as possible using agents. Yeah, yeah. So it’s a great, great reference source. And the visualizer too is what we were just talking about. So definitely check that out. And then the copilot app. I love, love, love the copilot. It’s worth it just for the canvases. For me, it is because you can build the canvases from scratch, whatever you want to have. The canvas looks like it does. We’re just looking at web pages, but you can make it simple. HTML is what we’re essentially looking at, but you can make it do whatever you want to make it

1:31:55 help you when you’re doing your work in a repo. Also, I’m going to put another link in. We’ll put it in the notes for the YouTube page is my team has actually built what we’re calling an interview coach that uses the SDK as a multi-agent handoff. So like you have somebody who takes it in and schedules your interview and you can do a, reads your resume or reads the questions, stuff like that. So it kind of show you in real time what I’m going to make them switch it over to this fantastic. Use this as well. So yeah, very cool. Thank you. Patrick. This was great. I

1:32:27 learned so much. Yeah, it was wonderful being with you. Oh, great. Well thanks everybody. Let us know what you think. Put it in the chat. We’d love hearing all feedback and great, wonderful. See you next time. Thank you all.