Which AI model is right for the job? Inside Kiro's model selection

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: Which AI model is right for the job? Inside Kiro’s model selection · Channel: Kiro · 57:22 · Watch on YouTube

Topics: AI agents, Task and workflow automation, Software engineering with AI, Model inference and optimization, AI models

Summary

Kiro team members discuss how to choose AI models for different tasks, balancing capability, cost, speed, and personal preference. They demonstrate model selection in Kiro and explain how workflows can assign different models to planning, implementation, review, and documentation. The conversation also covers Auto mode, model availability and lifecycle, experimental releases, and enterprise controls. The central advice is to use detailed configuration when it helps, or let Auto handle model, reasoning, and reliability decisions.

Key points

Kiro offers models from multiple providers, including frontier and open-weight models, so developers can choose according to their needs.
Auto mode can select a model and reasoning level, while handling some availability and reliability issues through retries or fallbacks.
Model choice involves trade-offs among capability, cost, response speed, and reasoning effort.
Different stages of a task can benefit from different models—for example, a stronger model for planning and a faster, less costly model for implementation or documentation.
Kiro workflows can orchestrate task steps, assign models to individual steps, and support sequential, parallel, and looping flows.
A fresh context and an independent model can make code review more effective.
New models may first be offered experimentally while Kiro validates quality, capacity, and compatibility with its features.
Models are kept available while supported where possible, but may eventually be deprecated when providers end support.
Enterprise administrators control which models are enabled for developers in their organization.
Credit multipliers provide a comparable way to understand the relative cost of models.

Chapters

Start	Chapter	Summary
00:16	Introductions and the model-selection question	The host introduces the Kiro team guests and frames the session around choosing the right model for the job.
02:57	Personal model preferences	The speakers share when they use Auto, specific models, and different providers for coding, research, reviews, and writing.
07:07	Matching model capability to task and cost	The host gives examples of using a smaller model for transcription and a stronger one for interpretation, while the guests discuss choice, cost, and speed.
14:50	Choosing models in the Kiro interface	The guests demonstrate model selection, credit visibility, and effort settings in Kiro.
19:28	Workflows and task-specific model choices	The discussion introduces workflows as configurable recipes for orchestrating planning, implementation, review, and documentation.
26:42	Assigning models to workflow steps	The speakers explain how workflows can use different models for planning, coding, independent review, and changelog writing.
32:25	Running workflows in sequence and in parallel	A workflow demonstration shows task execution, while the speakers discuss parallel work, loops, and independent background sessions.
35:58	How new models are released	The guests describe Kiro’s approach to adding models quickly while validating feature compatibility and quality.
41:14	Model deprecation and launch-day work	The discussion covers model end-of-life, the unpredictability of launches, and the automation used to test new models.
48:32	Workflow results and credit multipliers	The team reviews the completed workflow and explains how credit multipliers communicate relative model cost.
52:09	Documentation, enterprise access, and Auto	The host points to model documentation and enterprise controls, while the guests summarize how Auto selects models and handles reliability.
55:18	Wrap-up and closing thoughts	The speakers recap the choice between granular model configuration and Auto mode, invite feedback, and close the stream.

Quotes

The short answer is probably Auto. — 03:27

Sometimes you questions pop into your head for like random things like something about geography and all. You don’t want a model to really spend time reasoning — 13:17

I want to start with a clean fresh context window when I go and do that that review portion. — 30:19

The overwhelming majority of of the models that I’m excited about and reading about are are available within a day. — 47:29

they don’t have to worry about like what reasoning level or some of these parameters to pick. — 54:47

Concepts and entities

Kiro, Auto, workflow, Opus 5.5, Sonnet 5.5, Haiku, GPT, Anthropic, OpenAI, Brian Beach, Sujay, Akil

Full transcript (original language)

00:16 [music] >> All right. Hi everyone and welcome back to the Kiro live stream. I’m your host Brian Beach and it’s been a couple of weeks since I’ve been on. So, glad to be

00:48 back and hosting another one of these sessions. This week we’re going to talk a little bit about the the different models that are available in Kiro and a bit about how to how to make decisions around when to use what model. How do we all decide what model is right for the right job? And so, I have a couple of guests. We have Sujay and Akil, both from the Kiro team. They’re they’re both working on the Kiro product and so, let’s do some quick introductions. Um I

01:19 guess I’ll kick off. I’m not on this often enough that everybody knows me, but again, my name is Brian Beach and I am based out of North Carolina. Been a developer for more years than I care to count, but super excited about everything that’s happening with Kiro and I’ve been a Kiro user since the the launch just over a year ago. Um Sujay, do you want to jump in next? >> Yeah, sure. My name is Sujay. I’m a senior SD on the Kiro team. I mostly work on the Kiro back end. So, the topic for today about the AI model selection

01:52 with Kiro is is something that I I daily on. And yeah, really happy to be on the show. Thanks, man. >> Excellent. Uh okay. >> Hey, first of all, thanks Brian for inviting us to the show. Uh I’m an engineering leader with the Kyro team. Uh I’m based out of Seattle. Uh I’ve been with Amazon for quite long, maybe more than 15 years. Uh so, again, currently working with the Kyro team just to see like again how we can generally improve the Kyro backend and generally about like the model resiliency, model availability, and other things. So, really really glad to

02:23 share some of the things that you’re working with all of you. >> Excellent. I I didn’t quite catch that. How long have you been here? >> I’ve been more than 15 years with Amazon. >> 15 years, okay. Wow, wow, that’s a that’s a long run. I’m I’m just coming up on 12 any any day now. I’ve lost track. Maybe it is 12. But if if it is, it’s by a day or two. So, I’m just coming up on on my 12 years. I thought I was here a long time. 15 years crazy. I That’s what I thought I heard, but then I thought I might have heard it wrong. >> [laughter] >> So, cool, great great. Well, uh so so glad to have you here. Um I let’s let’s

02:57 kick off a little bit. We want to talk a lot about models today. And And I think this is one of the really important things that that Kyro brings is just uh a really broad selection of models. You’re not limited to models from any one vendor, right? We’ve got all the models from Anthropic and things from OpenAI. We’ve got a bunch of open weight models. Um what are you using? And And I’d love to hear from anyone listening to or watching, I should say. Uh feel free to throw it into chat. I’d love to hear what’s sort of your daily driver?

03:27 What what model do you spend most of your time in while you’re working? >> Yeah, uh I can go first, I think. Uh so, uh the short answer is probably Auto. I think Auto is sort of my favorite because uh it’s it’s the one that you would want to really have to think about, and it works in almost all the cases that you want to use. And uh though some it I do uh as you know Kiro has multiple tools now. We have the CLI, we have the IDE, we have Kiro Crew, we have Kiro Web. So, I

03:59 do find that sometimes on some of these platforms I have a slight little different preference based on how I am using that tool. For Kiro Crew for example, I use it for a lot of coordination task that and maybe a computer use. So, so maybe I wanted to pick a specific model there. But, but apart from that, yeah, like if I’m just chatting or like general conversations, Auto works really great. >> And I mean, that’s really the value of Auto, right? Is that I’m not spending all my time trying to figure out what

04:31 what model should I pick here? It’s like I’ll turn on I’ll turn on Auto and get get the best model for what I’m trying to accomplish. Akil, what are you what are you doing most of the time? You also have are you on Auto most of the time? >> I think it depends upon my task, but most of my crew runs on Auto. But, I think like when I’m using some of my IDE for some of my research task, I think Opus 5.5 have become a bit more of my favorite. I like maybe it doing a bit more of a explicit work for me. But, for the regular task and other things, I just mostly fall back to Auto.

05:03 And then, I also like again some of the GPT ones out there. I sometimes use it for like again an extra review and other things. So, I get Opus to maybe do my initial research and maybe like get GPT to maybe like review and other and say like again like what is its verdict on about the things. >> Sure. Sure. And there um I’m curious, are there times where you tend to switch models and and I’m going to throw one out there and and I don’t I don’t have any data to back up what I’m about to say. So, this is just

05:33 entirely my preference, but I spend most of my time with the Anthropic models when I’m writing code. But, I like the documentation that I get from the GPT models. So, if I’m switching to writing docs or if I’m writing a blog post about something that we did, then I tend to go the open AI route with the GPT models and prefer I just prefer the writing style. And I think that’s >> Yeah. >> There again, there’s not a lot of data to back that up. That’s just my personal preference. I like the I like the way it

06:04 writes prose. >> And I’m also like again, I have a feeling so lately getting GPT writing over maybe like it. But but I think Open 5.5 has gone a long way. I think I started liking it a bit more. I also like recently started playing a bit more with the open weight models. I think we have an Okami on uh Kiro. So, again, I started trying it out. It’s too early for me to share like again like where I will like devote some tasks to it, but I think I have been also trying it out a bit more in my free time. >> Yeah. Yeah. >> Yeah. >> Yeah, and I think that’s sort of the awesome thing with Kiro is uh hey, you

06:36 we have I I think a lot of these your preferences are up to you like you subjective and you like what the model does. And yeah, and you have the freedom to choose. So, so yeah, I think that’s the fun part. >> Yeah. Um I also think and I think we’re going to talk a little bit about about sort of using different models at different points in the flow, but there’s a lot of times where I can just go to a less expensive model. I don’t always need the very best model to accomplish what I’m doing. And and I’ll give you I I use I talk about this way

07:07 too much, but but my kind of side hobby is genealogy and family tree stuff. And so, I’ve got sort of a flow set up that I’ve been using for months where um I use I use the the LLMs in in Kiro as even just my OCR and things like that. So, I’ve got an old document, right? An old birth certificate or census record or something. I’ll put that into into one of the lesser models, right? I’ll throw that into like Haiku and let it go through and say, “Hey, do the do the translation of this for me.” I don’t need to a big heavyweight model bring to bear to just go and read a

07:37 document. But then I’ll take the output of that and push it into maybe Opus and say, “All right, now I want you to make sense of this and fit it into the story of these people’s lives that that we’ve been building over the last few months. That’s kind of a non-developer example, but there are plenty of dev examples where we can follow those same threads and say all right, for you know, I want maybe the Fable to be doing the planning and the architecture of this and then I’ll hand off to some other model to to do some of the the testing and and those kinds of things.

08:07 So, um it’s awesome that that you just have all of this choice. And I think that’s really what what we want to start by talking about is is that developer pain of of kind of trying to make those decisions. Um we’ve talked a lot this year, if you’ve been you know, if you’ve been on social media or you’ve been out on Reddit, you’ve probably read all of these threads about the you know, what’s the latest model and what model is everyone using and everyone’s moving from this model to this model. Um but let’s talk

08:38 about it a little bit like like where does Where does Miro fit in and and how do you think about all of kind of juggling all of this model choice? What’s What’s driving What’s driving that conversation out there and and how are we helping to address it? >> Yeah, I think like one thing which at least like again maybe I’ll take a lead on this thing. I think like one thing at least we are observing is like there is a plethora of models out there and the frequency in which newer and newer models coming out and it’s very hard for somebody to even choose like what is right model to use and across like different families,

09:09 right? And I think like that’s one thing where I think as a Miro product which we are trying to This could be like a good catalog of all the models out there and then we so that people can try it out and lot of these benchmarks and other things like I think I think it’s really hard to maybe choose or like again say like this is the right model for this task because it’s really hard just from a simple benchmark to say like okay, everything will will be best with one class of models and some also goes to I think what we talked a bit earlier about there are personal preferences, personal styles and some things which we like better than maybe

09:40 something else. So, I think as a Kiro product, we wanted to give that offering like so people can choose, people can try it out in one place, and then also like again shift some of their workflows and other things more to their preferences rather than like needing to move across different products to like try it out. We try our best to like I think bring the all the latest models on Kiro. And also like again, we are trying with our auto systems which I was mentioning earlier where sometimes you don’t really need to bother about any of these things, right? I really want to focus on my business logic or what I’m building, and I wanted to like offload that overload of like

10:13 which model to pick and other things, and then let let Kiro decide that for me both in terms of availability or resiliency because that’s another thing we are seeing right now as GPU capacity and other things as case. I I So, I again, I personally feel like auto does a good job because it takes care of some fallbacks, other things and all for me, and I just like maybe like leave it to auto to take a lot of those decisions for me. >> Yeah, man. And I’m seeing confirmation of a lot of that. I’m going to assume this is Nirmal Patel if I’m finding the right split in in that username, but

10:43 talking a little bit about how auto is is sort of the the default router and and and the impact on credit saving saving a lot often. Especially over some of the the newer models can be quite a bit more than than auto mode. So, I think if if I reflect on what I’ve seen in the past year and the discord out there, it started at the beginning of the year very much with all of this stuff about model choice, and and there was every week it was about, “Hey, this model’s outperforming this model.” And and you saw these big shifts

11:14 in market even even across full vendors, right? People going from from Anthropic and switching over into OpenAI, and then moving back depending on the model. Then as we got to the second half of of this year, a lot of the conversation has really shifted to cost and looking more at not just what is the best model right now, but like where where do I actually want to land in that capability cost curve? Are you seeing the same thing and thinking about the same things as you’re building this?

11:44 >> Totally. I think like again when we talk to customers, I think I think like that’s one of the common ask which we are generally saying right like how can I control my cost or how can I make sure that things are within the budget so that it’s not just the free token thrown out there. How do we make sure that like we get the right value or like the right output for any of these usage. So I think like again you rightly mentioned like auto does a good job in making sure we we again like cost it like in the in the best case other than like again speaking or choosing the individual models out there and that’s a feedback

12:16 at least we are getting from many of our customers that like we speak auto because we can trust in terms of the cost and the reliability out there. >> Yeah, absolutely. And are you still often thinking about sort of are you framing this yourselves as like I just want to be on the best model regardless of the cost or or are you also internally starting to think a little bit about those trade-offs and and finding the right one? >> We we are internally thinking a lot about those trade-offs because things are becoming more and more constrained or I think everybody are looking at like what’s the real value value for the money there. So I think in general as an

12:47 organization we have been thinking about like what how can we really like make sure we get the best value out there and what how do we pick the right models and other things for it. Yeah. [snorts] Earlier in the year I think as you rightly mentioned it was more about try everything out, see like again like what is the magic out there right? But right now I think I think it has shifted a more towards like how can we basically get the best value out there and all. >> Yeah and I think along with cost there is also this an aspect of like speed like how fast you want the model to respond and you in some task like

13:17 for example I think you have have an example about genealogy. Sometimes I questions pop into your head for like random things like something about geography and all. You don’t want a model to really spend time reasoning and it’s a quick question and answer. So, so there are models that you can use to have that really good back and forth conversation and you can that’s also usually come with a lower cost. So, I think so there is definitely as you rightly said like there’s a lot more nuance now in how we want to think about these and also what what your

13:49 preferences are in how the model responds. >> Absolutely. Yeah, and there’s there’s also so many knobs and levers these days, right? It’s not just which model do I want to pick, but you’re also thinking about hey, do I want to be you know, how how much thinking do I want it to do before it executes? Do I want it to think for a long time? Um, and and there are times where I’m even I can be frustrated with it and say hey, like like this is not that hard a question. Why are you still thinking? Why are you know, this one should have been easy. At least that’s the way I perceive it. So, um, it is it’s getting

14:20 it’s getting harder and harder to find that that right balance which again, I think comes back to sort of auto mode and and why that’s really valuable. I don’t want to be spending my time thinking about all of this. I want I want Kiro to take care of that for me. >> Yeah, and I but I think we also see like I think different personas of customers. Some really wants to really get all the knobs, right? They really like want to be in control of all those things. So, we want to explicitly give that option for whoever likes it, but some again like now I don’t want to really think or take care of those things. I really want Kiro to take care of it for me and like

14:50 maybe they have they can just like fall back to auto and then get Kiro to take care of all those things for them. >> Mhm. Yep, yep. Excellent. So, I think um, maybe it would make sense. We’re talking a lot about this in the abstract. We want to jump in and and look at it in Kiro? >> Yeah, sure. We can do that. >> Excellent. Let me uh, let me share your screen for you here. Yep. Mhm. Excellent. >> Yeah, so this is a typical

15:20 Kiro Uh is the Kiro ID, so this is what you would typically see. And uh as you can see here, this is a completely new session. So, uh this is what we are talking about. Uh Kiro provides uh um like a bunch of these models listed across different providers. We see both Anthropic and uh OpenAI models here, and also uh we can see other open-source models as well. And then uh the the and uh it gives you a visibility

15:52 into like what the cost would be in terms of credits for you. And I think we talked about uh uh Let me try Yeah, so we talked about uh effort, right? Like so, if you choose uh specific model, then you can also choose what kind of effort you want uh the model to reason with. And then uh that impacts both how deep the model goes into a particular area before it responds. Uh and also it uh impacts uh

16:22 how much time it takes, and also how much token it consumes. So, uh so this is uh what we want Kiro to provide is the choice is, “Hey, if you are if you if you are if you really knew your use case well, and you really prefer a certain type of behavior, and you know that this is a deep task that uh you want the model to really think about before responding, you can choose a model that is appropriate to the task. Plus, you can also uh tweak the effort level, uh and make the trade-off yourself between

16:52 what the uh Yeah, how how deep the model will go.” So, uh And uh Yeah, and but if you, for example, yeah, the I think we’ll talk about auto a little bit later, as well. But yeah, if you if you do not if that’s not something that uh is going to make a difference for you, then you can simply choose auto here. >> Excellent. Yeah, and there’s um Um, I mean, maybe just scroll through the list there. There’s There’s a lot here. >> Yeah, there are a lot of models available here.

17:24 Yeah, including Kimiko 3 that we made available earlier this I think this month. Yeah, so that’s pretty fresh. >> Yeah. Excellent. And so, essentially what’s happening when I pick one of these is is I’m setting the the model for that conversation on that tab. >> Correct. So, that’s right. So, if you select a particular model, then that’s a more conversation you’ll have in this one. Uh let’s say

17:54 this project. So, this is a project that we are going to look at a little bit later as well. But, you can at the same time have parallel conversations which might be doing more deeper things and or maybe more easier things and then based on that you can actually have a different model selection as well. So, you can have threads in parallel with different models and all of them are custom to whatever workflow you’re running currently. >> Excellent.

18:25 And I think this is probably evident, but I know I’ve heard some confusion around this that maybe this isn’t true of of all the tools out there in the world, but I can change models halfway through a conversation, right? I can be kind of cranking along in one model having a conversation and then decide that maybe I you know, I’m coming into something where I don’t need this much power or I need a little more smarts or I want to switch over to you know, in my case say to an open AI model because I’m going to work on the documentation now and I prefer the

18:56 output of of the GPT model. I’m not necessarily bound to that for the entire conversation. >> Exactly. So, you have the complete freedom to switch between models in between your conversation. So, yeah, if I at this point I want to switch to a different model I can do that and it will happily check along and same thing for the effort. >> Excellent. Awesome. So we we maybe touched on this a little bit

19:28 but let’s talk a little bit about you know, how do I make those decisions around what is the right thing for the right job? Is is do you have any guidance to users out there about either what to pick or or maybe how to profile that decision and get to a good outcome? >> Yeah, so I think it it’s the right tool for the right task, right? So I think that’s how at least I think of it and what I find is that there’s a general pattern where I want

19:58 maybe one of the more advanced models to plan out things for me that hey, I give it a give it a very general maybe even vague task and it can I can have a conversation with it to drill down to the actual requirements. And then and the Opus for example is actually really good at that. And then if you once I have a good plan and the actual coding implementation can actually be also handed off to a smaller model as long as they’re in the right in the

20:29 right chunks. So so and of course there are other tasks that you can even do with small models like writing documentation for example, you can also maybe switch to something like a high cooler there. This is one of the things that uh we at least works for me and I think this is one of the things that at Kiro we are also making it easy right now for users to do and that leads us into maybe like I can demonstrate maybe the next part of

21:00 it what how is a good way to package it which is through a feature that we launched again quite recently which is workflows, right? So with workflows just to give a brief overview of workflows, workflows are recipes that you can configure and you can share with your team and the they are basically the recipes basically contain steps that you can ask your agent to execute and it’s actually super interesting and deep topic in

21:32 itself in how you can how you can parallelize workflow steps and stuff like that but I think for our purposes we can pick a simple workflow here that is simple but frankly it is very close to a dev workflow that I would actually run. So here we can see >> By the way, we have not done workflows on on the live stream yet. So this is good. Like you know, feel feel free to spend some time here and and dig a little deeper than maybe you would just to talk about the the model aspect of it because this is the first introduction

22:02 to anyone who’s been watching. >> Correct. So So generally what you want that you as a developer, right? Like you have this very natural workflow where you as I was saying you maybe plan ahead on what the change is, maybe help the agent to formalize or remove ambiguities from the overall requirement and then you go and actually implement the code and then obviously you want to

22:33 review the code before you commit or to ship it and then you want to update documentation. So this is a very standard flow, right? And in generally how most people would do it or I would at least do is that I am the one who’s orchestrating that workflow, right? Like as a I am asking the model to ask to talk about planning and then once it I am satisfied I’m asking it to go ahead and implement it and then once it is done to review it. But with workflows you can

23:03 actually automate a lot of this orchestration within Kiro itself. So that the agent will it will actually orchestrate and it will only come to you for permissions or for any specific issues that it runs into. And this can actually as I was saying become be is very powerful because you can maybe run multiple steps in parallel, you can have loops so that you actually if you want to go deeper into an issue or like you know want you wanted to find

23:34 issues and then come back and fix them. You can all plan all that in a workflow. So so it’s a very powerful tool I mean it’s a tool. >> And it will it will show us this in sort of a preview mode, right? I know you’re kind of looking at the source code behind it, but I also have a >> Correct. >> Yeah. >> Yeah. >> Um I probably you know for someone new who’s first seeing it this is this is maybe a little easier to digest. Um >> Totally. >> So most of us can just translate JSON in our head at this point into a picture, >> [laughter] >> but >> Yeah. Yeah, JSON and markdown. But yeah, so

24:06 that’s where we live right now. But I think yeah, we have a you you actually can see exactly what the steps are and what goes into as prompts for each steps and what what what the expected outcome would be from from the those steps and yeah, what what are the next steps in the flow. So >> And and then you you’re sort of I I think you implied this earlier, but I understand that there are some you know control flow constructs in here, things like looping constructs and

24:36 branching constructs and things like that that I can build into these workflows. I know you you want to get to the part about models, but >> Right. >> Maybe maybe let’s fill in some of those gaps for everybody so they understand it. >> Correct. So you with workflows you can actually introduce uh you can I think usually what you want for a flow again for a flow is this is a simplified version is that you implement, you review, the review will find issues, you will go back and have the implementation agent uh actually

25:08 look at the comments and fix it. So, that’s something that is uh you can do I guess you can set up a workflow to do that and you can set up uh hey, I want you to run the review maybe at most uh three times. You don’t maybe you don’t want to uh it to keep uh it to get stuck in the loop. So, you can also like base have uh those uh short uh circuit breakers essentially and uh you can also have parallel flows. So, uh again, so it’s a very it’s a it’s a new set of tool uh that we are actually

25:40 providing with Kiro now and I’m really excited to see how people will uh how folks will build on top of that. So, uh yeah, I think at least for me it has been a uh it has been a game changer on how I can actually think about orchestration uh and yeah, basically taking myself out of the orchestration and giving that also more to the agent. >> Yeah. It It seems like nearly everyone’s kind of talking about software factories these days and uh you know, this is maybe a an overly simplified version of that that we’re looking at today. But,

26:11 this is how you’d put that together, right? I I want to go through these series of steps to get from an idea all the way to um production or or maybe you just take it to a PR and and then you have a human get involved depending on your risk tolerance. Maybe you take it all the way out production. Um but, these this is how you would then go and map that out uh so that Kiro could orchestrate it end-to-end for you. >> Absolutely. Yeah, I think uh that’s that’s yeah, this is getting us much closer to that now. >> Yeah. Excellent. Um so, sorry. I know I I kind

26:42 of took you down I I forced you down this path of of giving us a bit of an overview on workflows. Let Let’s take it back and connect the dots, right? The The point of this being, hey, now that I have this workflow with multiple agents defined in it, I can start to map the right agent to each of these phases and decide what to use where. >> Exactly. So, this is what this is what I was talking about earlier is, hey, I generally I find that that I I

27:13 need the bigger models to actually help me plan and, you know, look ahead and because they are the and also maybe reason more about what I’m asking before actually implementing. So, so this is so workflows are a really good fit for that. So, this is where it going back to the JSON helps is you can define steps where you say, hey, this is a plan step where I only want you to do read-only planning and I want you to investigate

27:44 the code base and actually understand what happened. So, one thing about the workflows I didn’t mention is you also and I’ll I’ll demo this right after this is that when you you also give a prompt a prompt when you start the workflow. So, you can tell it, hey, what should be the action or task that wants to investigate and then it will take that task and it will run the steps in context of that task. So, but the sort of the

28:14 relevant thing for this discussion is that you can then specify a model for each of those steps and these could be different models. So, generally my preference here is that I’ll use an Opus for the actual planning and then I’ll switch to something like Sonnet, especially 5.5 is like a really good I find that it’s really good at coding and if you give it a contained task, it it’s actually also really fast at coming back with code. So,

28:44 that’s what I prefer like generally Opus 5 can obviously do the same thing, but I’ll be spending more uh credits, but at the same time it’s also going to take more time for it to get ready versus with Sonnet, maybe I can run more iterations in the same time with with Sonnet as it finishes the actual coding task faster. >> [snorts] >> Then I think we want we have a reviewer task. So for this one I generally if I sometimes I also use GPT models for

29:16 coding. I do find again subjectively for some tasks that like maybe for configuration changes and stuff like that. I find the their the conciseness from those models a little bit more appealing, but that’s just my personal preference, but I think it really helps to switch the it helps to switch the provider so that you have a completely model trained on different things that are just looking at your code and all of these start with fresh context. So they are actually

29:48 looking at the pure code as if like a reviewer is looking at your code with with only the context that they see in there. >> And I’m glad you said that too cuz this is a really important part particularly for a reviewer. You want that reviewer to have a fresh context. Otherwise it ends up being tainted by the decisions that were made earlier. It’s like when I write code and I review it, it’s really hard to do your own code review. >> Exactly. >> Cuz you you’ve made all these decisions. You you know, you’re you’re committed to them. You’re not seeing the things that are wrong. Where I hand it off to

30:19 another person, a second person to come and do that code review, they see things that I wouldn’t. Even if you’re not changing models, I think it’s always best practice to say, I want to start with a clean fresh context window when I go and do that that review portion. And then layering in a different model not just you don’t know anything about what the decisions that I made, but also you’re a completely different model, a completely, you know, different person, if you will, doing this review is really valuable. Um right there I think the first example is

30:50 and I do this myself all the time like when I write whenever I’m writing I can’t see my errors. I have to come back I have to give it like a day and I read it the next day and I’m like, oh, look at all these errors that I didn’t see yesterday because I’ve sort of, you know, I’ve lost some of that context that was stuck in my head. But handing it off to another person as you’re doing here is is super important. >> Yeah. And but then just one thing to add I think what I found really effective is like doing the front planning of like not asking to implement right but do the front work and switching into smaller task. I found it really effective right because if you ask the same model to do

31:21 everything but at least like again giving everybody a small concise task and especially like Sonar and others right which you can go and really execute on it. I found it really effective in some of my my workflows and my job sites at least I used to do. >> Yeah. Yep. Absolutely. Um >> Yeah. Yeah, so this is I agree with I think both the points there is for reviewer definitely helps to have a second sort of independent opinion that’s what you do in a normal PR flow as well.

31:52 So this is this is a good way to mimic that kind of flow and with using a heavier model for model for planning you can actually break down the task before you get into the coding bits. Um Yeah, so getting this flow so ultimately I think once the task is done we want to update the change log for this uh package >> Sure. >> and that’s a very simple task of just reading the diffs and coming up with uh with this as nicely worded change log so Hayku is especially 4.5 is more than

32:25 capable of doing that and with and and it it actually is really fast and again like cheap to in terms of credit that it consumes. So so this is what the workflow Yeah, this is what the work the workflow would look like, a simple form. So, and we can see it execute here. Let’s say So, this uh project, just to give you some context, is a

32:55 is is is a let’s say a mock control plane. Uh this is kind of like what I work on day-to-day with the Kiro’s back end. Uh so, this is a fairly typical task that you would ask an agent to do. Uh maybe on the simpler side here, but uh we can we can see how uh workflows would handle that. So, with workflows, uh you can see uh both the runs as well on the it actually implementing, and then uh it will walk

33:25 through, rather it will run through each of these steps, and then uh you can also dig into what it is doing. And uh so, we in the plan step right now, if you see, it is just reading through the files, trying to get the context built, and then uh it will hand off to the next tool, uh next uh step once it is done. So, we’ll let it let it run in the background. >> And this this particular workflow is is a series of sequential thing. This is sequential versus multiple parallel, but we could also now the the other thing

33:56 I’d be doing is workflow is fanning out. >> Exactly. >> Maybe this is a terrible example, but if you wanted to add a bunch of fields, you might launch a different worker to do each of them. Don’t do that. That’s like you’re you’re touching the same code base, and you probably shouldn’t You wouldn’t ask five humans to go add five different fields to a form. But, you know, conceptually, I think you get the idea, right? We could fan out and have things running. >> Exactly. So, you can fan out, you can uh you can break down the task, and then you can also have loops uh so that it if you if you find something is broken,

34:27 it can can come back and maybe even re-plan uh from the start if if some of the assumptions are incorrect. >> Yeah. And and while you’re doing that, that that chat session that you’ve got going on the right, um, right, that’s all background. So, I can continue talking to to Kim as well. That’s all as well. >> Exactly. So, you we have you can have independent sessions that are running here that uh, can actually uh, yeah, you you can again choose a different model for these sessions and

34:57 it’s completely independent of what the workflows are themselves running uh, in parallel. >> And I think probably everybody has come to understand this at this point, but the other nice thing about running all this as background tasks is I’m not filling up that the the context window on that primary conversation that I’m having. So, coming back to my example of running OCR on a on a census record or something, that is something that uh, would typically blow up the context quite a bit with data that I only care about three people on that form. I don’t

35:28 care about everybody else that was living on that street at the time. Um, but if I can push that into a background task and I just get back the list of names that are important to me, well, I’m not putting anywhere near as much overhead into that context. >> Exactly. And as you can see here, like I think you could basically be doing another task, but in parallel in the background it has finished planning and it’s then moving on to implementation and yeah, that’s completely independent in an independent context window. >> Yeah.

35:58 Cool. Excellent. >> Mhm. >> Um, >> We’ll let it run in the background. >> Yeah, yeah, we can just keep chatting while this is happening. If if you’re good on this, I’d like to talk a little bit about sort of how how you as the dev team on this are thinking about models and and kind of model life cycle. When when are we bringing new models in? I don’t know that we’re going to be able to answer this question out of chat, but there is sort of a chat here about like when is Kimmy K3, which we do support,

36:30 going to be available in India. Um I I don’t know what the I don’t know any of the uniquenesses of uh if there’s any limitations of India specifically. But let’s talk a little bit generally about about kind of the model flow. Uh what I’ve seen is at least for for the Anthropic and now GPT models is as soon as we add support or as soon as AWS has added support, then um there’s usually a really fast follow, often same day and if not within a day or two after

37:03 it becoming available. >> Yeah, I I I think we try our best to like get the models as soon as possible to on Kiro. And I think again we see a lot of excitement where uh people want to try it out. People And people also see like they again every model comes with like a good amount of promises, either better efficiency or other things. So we really want to make sure that uh people can use it within the tool. Uh but at the same time we also want to be careful that it works along with our feature sets, because I think like again uh in general like we uh support like steering other ends. Like we want to make sure features like hooks and all

37:35 those things work because we don’t want to get a bug out. So we do a lot of automations to make sure that we validate, we make sure that the quality is there. Um and lot of times like what we are doing now is like uh we get the model as soon as possible out in experimental mode. Uh we we uh for that at least like again people can try it out. Um as soon as at least to to a reasonable amount of confidence we have, right? While we kind of like maybe tune further on the end in terms of quality and other things. Uh and once we are really fully confident, we then move it back into like a a GA

38:07 mode, right? So people can really use it in their regular workloads. Uh so that’s at least like what we try to do. So uh we try to move as fast as we can, but at the same time we want to also be careful that it it works and then we are not like again like pushing out anything out there. >> I I think generally too >> just to to maybe dive deep into this a little bit, when a new model first launches is and is in that experimental mode, A, you’re putting it through its paces, right? Making sure that it is performing the way the way you want it

38:38 to, probably making some tweaks I assume to some prompts along the way. Um, but then I know there there are also some other things that we’ll call out in the documentation about um, the the I’ll say Chris here, right? And then explain what that means, which is uh, when a model has has made it to kind of non-experimental status, when it’s gotten out of experimental, we make some guarantees around the locality. So, for example, right? If if you’re running in the US, um, all of that inference will

39:09 happen across the the US regions. So, it doesn’t necessarily always happen in in US East, it’ll fan out because we need capacity often. Uh, same thing happens in the European regions, it fans out across a few regions within the EU. Um, when we’re in experimental mode, often we’ll be in global mode and just saying, “Hey, we can use inference anywhere while we’re building out the the capacity.” Because we’d rather be in experimental mode for a little while than than you as a customer getting a lot of errors saying that we don’t have the model capacity for this one right now. Um, and it takes a while, right?

39:41 You don’t you don’t know when you first throw something out into the world, how how quickly are people going to move to it? If I put 5.5 out there, are people going to stay on five? Or is everyone going to immediately move to 5.5? And you’ve got to kind of think through some of that balancing. So, that’s another part of that experimental mode that you’ll see for a little while, and then it’ll we’ll take the experimental tag off once everything’s settled down and we get a sense of where it is. >> [snorts] >> Yeah, I think exactly. So, we want to see real-world usage. So, so in terms of the life cycle, we do try to predict what capacity we need up

40:12 front based on past data. But again, each model is different and and users find different ways to innovative ways to use the same the new versions of the model. So, we use that time to collect some data on on if if we have to make changes. At the same time, even before we launch the model, we we we spend a lot of effort in making sure that it it works well with Kiro. So, we the harness and the model generally

40:44 have to work together. So, in this case, Kiro we figure out if we actually have to make changes to the prompts or we have to change how the how tool looks tooling looks for the particular model so that we can maintain the quality. This is based on sort of our own benchmarks and what we know from past and that helps, but also like just getting data from actual users and their usage is is obviously can’t replace them and that’s what we use the experimental

41:14 phase for. >> Yeah. And I think when we talk about model life cycle, we also have to think about end of the life as well. We try to keep the model as long as I think it’s mostly supported by the model providers. I think we recently went ahead and deprecated Sonnet 4 I think that was like end of life from from an Anthropic support perspective. But I think I think like we try to keep it as long because not everybody wants to keep changing their workflows or their automations around it. We want to support like as long as we can. But at the same time like again like as workload and other things move off, we

41:44 want to make sure that we are also using our capacities and other things in the most effective way. >> Yeah. Yeah, I just saw the Sonnet 4 end of life announcement. I I think is that the first? >> I I think so. At least to my knowledge, it is the first we went through, but I don’t know whether there was something before that. Yeah. >> Yeah. And I assume I mean that’s going to keep happening now, right? We’re you know, Kiro’s been Kiro’s been out now for maybe 15-ish months post GA that that we’ve got a runway and so we’re starting to get to the point where these models are coming coming off the other

42:14 side. It’s going to feel like like any other service where things are different frameworks, different run times are starting to reach end of life. We’re going to start thinking of models that way as as having a full life cycle and needing to to move them off. Um I think in this case I really doubt that anyone was still using uh Sonnet 4 heavily in any way. The I I feel like the next one I I think it was 4.5 was sort of like you saw that big jump, and then it was you know, as as we got those next couple were that

42:45 inflection point as things were really taking off and things really felt like they were changing. So, some of the next ones there might be a few people hanging on to, but I doubt anyone was hanging on to to Sonnet 4. Um Cool. So, you’re both on this team, and we’re talking a little bit about this launching new models. I’d love to hear from you, like what is that day look like when when one of the big model providers launches something new. I’m I’m assuming that you don’t have any foreknowledge of it, or if you do, you

43:16 know maybe a few hours before the rest of the market does that this is about to happen. You are probably in the middle of trying to get to a deadline as we always are, and suddenly it’s like, “Oh wow, drop everything. New model just launched. We’ve got to get on this.” What what does that look like behind the scenes for you? >> Yeah, to be very honest, this is one thing that’s a lot of unpredictability out there, and all of us likes to get a bit more of a heads-up these but this is one thing where the market in general is moving really fast, and we also really try to get along that line. But I think like

43:48 what we have done is like we have done a lot of improvements in this area by doing a lot of automations in terms of like the evaluations, how do we evaluate the benchmark, how do we do the capacity planning, how do we test the correctness in terms of feature validations and those things and all. So, there’s a lot of like I think art or like again work I think the team has done to like improve this whole process. But, that said, still uh not everything is completely backward compatible. Uh there are there will be like quirks and other things which we figure out as we they again like on the day of launch, right? So, there’s some on the unpredictability or like chaos out there

44:20 to make sure that it works. And that’s also the reason why we have the experimental mode out there where we can continuously tune it like might not be 100% but still because some of these corrections is sort of uh UIs and other things takes longer time, right? So, yeah, we’re also sequencing the way of like in a P0 or like what’s the most critical thing versus what we can done do in a bit more of a I try to tuning and other things. So, um we’re doing our best in terms of automation with a predictable way but there’s also a lot of unpredictability out there to be honest. Uh with different model families that we

44:50 support, uh it comes with its own nuances which may be like not every harness that we have, right? There is they only support maybe a specific class of things. We support a broader class of things. So, we need to make sure that it works with various of these things. >> Yeah, I mean I I imagine to to compare that to traditional software development, we’ve got this huge battery of unit tests. I’m sure they’re more complicated than a unit test, probably prompts with with expected outcomes and things like that that we’re going to run against. Um and that’s great but experimental mode is is a really important time when you’re kind of in beta

45:21 effectively with a set of new customers running real things against it and getting real feedback. The the those those canned tests that you’re trying to plan for, they only ever go so far. So, it’s it’s it’s a necessary period of time where we see how people are are adapting to it. So, yeah, I think I I cut you off and I know I took your screen off but you know, it felt like we were chatting but I can throw it back up when you’re ready. >> Uh no, that’s okay. I think the just one thing I wanted to add is I think uh the at least the team we have focused a lot on making sure that we remove any delay or

45:53 anything that from our side as much as we can. Uh there’s obviously be unpredictableness in terms of the models or what comes from the providers. So, we have spent a lot of time optimizing that pipeline so that as soon as we know a model is going to be available, we are able to get it to customers maybe in in a matter of hours, right? And I think like with any software deployment pipeline, we have stages. We try to make sure that we can test it ourselves and we can test against these the benchmarks that Akil talked about.

46:26 But the goal is that whatever we can control uh get it down to the minimal time uh required and then uh hopefully what remains is just what what unpredictably comes from the predictability comes from the model itself or from from the providers. So, uh so yeah, I think it it’s still I guess something that we are working on, but again, I think it’s really important to customers because it’s it’s really exciting to use a model that has recently really been launched

46:58 with new capabilities and uh that’s something that we uh want to do ourselves as developers, but uh but we can it’s what we want to do ourselves. >> But but I also want to add one thing. I think like one thing I I kind of always hear maybe from the some people asking, right? Why not I get it almost immediately? I think we try to keep a minimum bar. Like we don’t really want to like really get things out when we are not confident about uh that thing. So, that’s one thing we really don’t compromise on and we sometimes hold back, right? No, this is not ready to like even expose in an experimental mode

47:29 and we need a bit more time before we do that. And I think I think we are a bit more again lenient on that side. Again, the minimum quality bar which we think should be there before we expose to our customers. >> Yeah. But by the way, I I don’t think you’re giving yourself enough credit there. Um the overwhelming majority of of the models that I’m excited about and reading about are are available within a day. >> Yep. >> Or or often I’m playing with [laughter] them that night after I after I read about the launch that morning and I’ve got access to them. So, by by no means do I think that that you are not delivering here,

48:01 you know. Yeah, there’s always a couple odd ones, as you said, where where things take a little longer, but generally this goes really quick. So, well, kudo kudos to both of you. And the rest of your team for everything you’re doing behind the scenes. I can only imagine the not not so much the pressure so much as just the surprise of being hit with that. Like, “Oh, wow. Here we go again, right?” And and it feels like that must happen every week or two. >> Well, I think we are seeing multiple models a week now. So, I think it is more than >> [laughter] >> Crazy. >> Like really thing like yes, yeah. >> Yep.

48:32 Excellent. So, um So, yeah, I apologize. I took your screen down because we felt like we were just chatting, but I’ll throw it back up and I know we can we can kind of wrap up. I I that’s been done for a long time for anyone that’s that’s trying to watch the clock on this. This one finished quite a while ago. >> Uh exactly. So, I think if you see here, uh I’m trying to try and uh you see the session. Yeah, so if you see it based on it completed all the steps and as we were talking, right? I did not have to prompt it in any way, right? And

49:03 and you can also see in terms of the timing where it like spend time on. So, the larger model obviously for planning Opus took longer, but that code change itself was pretty small. So, uh the so both in terms of tokens and the time that was consumed here, Sonnet 5 5.5 did really well. Like it could make the change like really quickly. And then of course the reviewer took some time to build up context and understand if the change is correct. And then Haiku took almost no time at all to come up with

49:33 the simple documentation change that we needed. So, this gives you an idea of what you can do. Obviously, this is a fairly simple example, but even here I think you can you can see using you with workflows, especially, but in general also with Kiro, you if you have preferences for certain models, you can there are different ways to express them and then uh with workflows especially uh it it will take away a lot of the orchestration for you while uh adhering to those preferences and giving you these benefits of credit usage and time

50:05 as well. >> Excellent. Cool. Um so so you kind of ended there talking a little bit about credit usage. Um maybe for for those who don’t know it well, let’s let’s talk for a minute about those credit multipliers and and how those correspond to what you’re spending, right? Cuz obviously the the higher credit ones are costing you more. How does How does the the credit multiplier relate to what it’s costing? >> Yeah, so uh it ultimately comes down to how much

50:36 work the model is doing and how uh how many tokens it is generating. So uh based on uh that and I think from uh how from how Kiro also thinks about our our Kiro has seen with our benchmarks on what typical task would cost, uh we try to uh find the right balance where uh the users don’t really have to think about, “Hey, how much work is this actual model doing?” Uh and how much tokens is it consuming? And so the credit multiplier is a way to express

51:08 that in terms of the which is is easy to also compare models in with the credit multiplier. So it actually tells you uh uh both in terms of how expensive maybe a model is, but also how powerful it is, right? So I think uh that’s a balance we try to strike is uh we want to be able to uh again provide uh these models from different providers who might have different token uh generation characteristics, but there’s a uh simple way for Kiro users

51:38 to translate that into how much it would cost them. >> Awesome. Absolutely. Uh so I I just brought up quick the the documentation page for for those who, you know, don’t have access to Kira and aren’t looking at it. They are all um listed here. And uh you can at the bottom of it, I think, see the credit multipliers. Maybe they’re not on this page, they’re on a different one. Um also this is those laying out all the stuff that we talked around around the model life cycle. But if you’re interested in and want to

52:09 get a sense of what’s available, you can see all the different models here and and read a little bit more about what’s supported in Kira. Um And then the the other thing that that I wanted to know quick is I know probably most of our our viewers are individual developers and aren’t thinking about that that uh admin experience, right? Likely they’re not the ones who are governing this and rolling it out to their entire organization. But I did see the question earlier

52:39 about, “Hey, when is this model going to be available?” And it made me think part of what that what that might actually be is not that the model isn’t available, but that your organization, if you’re a developer within a larger enterprise, hasn’t rolled that out yet. And so in the in the management console here, and this is only if you’re running under an enterprise account, not an individual account, but you your your admins have control over what models are available to you. And so when a new one

53:11 comes out, like like Sonic 5.5 is fairly recent, and I haven’t actually remembered to come in here and turn it on. So this one hasn’t been enabled yet, and um I’m the only developer in my very simple org, but in a real-world organization, if the admin didn’t come in here and enable it, it’s not going to get rolled out to their team, even though it is available in the tool. But um so so as an individual developer, if you’re not seeing something that we’re talking about here, probably you need to reach out to whoever’s running the the developer experience team within your organization and ask them to turn that

53:42 on. Um for those who are on the admin side, I do want to call out that this exists, right? That that just because we’re rolling models out doesn’t necessarily mean that your organization is prepared to accept them and roll them out to your developers. And you have control over that and can make decisions about when and how to roll it out. So. Cool. Um quick quick time check. I think we are pretty close to running out of time here. Anything that we didn’t talk about that you want to?

54:15 Miss anything? >> Uh maybe one thing I would mention is I think in terms of auto we talked about uh that a it picks uh if you’re on a paid depending on the plan you are it picks a open class model. It also uh again based on our benchmarking and learning it picks the right reasoning level uh for you. Uh the third aspect of it is that I think we touched upon it briefly but it also uh it depending on the capacity picture or depending on the reliability picture of the particular model it also can

54:47 automatically fall back or retry your request so that you don’t have to deal with the failures yourself. So that’s those are the three axes that we think of in for auto is that we want to provide uh good experience to customers where they don’t have to think on any of these axes, right? Like they don’t have to worry about uh which is the right model, they don’t have to worry about like what reasoning level or some of these parameters to pick. And also if there is uh some unreliability temporarily at least that uh they don’t have to they they will not stop and they don’t get

55:18 stuck. >> Great. I I so I I think if I could summarize all of this for for those who want it there is tremendous configurability here across multiple model providers across um both frontier labs as well as the open weight alternatives that are out there. You you can pick amongst all of those and you can set up really granular, really complex workflows that use all of those and bring those to bear at the right time. On the other side of that

55:49 though, as as Sujay is saying, if you don’t want to deal with all of that, if you just want to go write code, um that’s when you bring Auto in. Right now, I think there’s a there’s a good one in here. LinkedIn user. LinkedIn doesn’t give us who’s who. It doesn’t give you the names of the people. But someone out on on LinkedIn is talking about this and and how um they tend to use Auto. That’s their go-to. Super convenient. And they’re calling out the student plan. If anyone’s not aware, there is a whole student plan for Kyra that you can get access to Kyra credits as a student. Um check that out on on the kyra.dev

56:20 website. But as a student, that’s probably not what they’re trying to do, right? They’re not they’re not interested in building out this complex um software factory. They’re just they’re they’re playing and learning a new technology and trying to pick up a new skill. And so, just turning on Auto is likely the the best option for someone like that. And And Kyra’s giving you both of those, right? If you if you want to get really in the weeds and figure out exactly how it works, we can do that. If you don’t care and you just want to go, turn on Auto mode and we’ll do that as well. Excellent. Um

56:51 Akhil, any any parting thoughts here? >> No, I I think you covered it pretty well. I think I think again, we want this to be the best product for any developers out there and we are always welcome to feedbacks and then how we can make it better. And thanks for all of you guys who are sharing because we are trying to incorporate and see like how we can like offer the best experience for all of you. >> Excellent. Great. Um I’ll throw up one more quote. Um I don’t think I could say it better. Kyra’s amazing. So, I’ll let someone else say it for me. But thank you both very much for for spending an hour with us. I really appreciate your time. Uh hopefully we

57:22 can have you back. And as you said, there’s there’s a new model every week, if not multiple. So, let’s let’s do this more often and talk about all the with that are happening and all the models that we’re adding. >> Yeah, there’s a lot we are cooking, so we would definitely love to come over here and share some of those things. But thank you. Thank you for having us. >> Excellent. Thank you. Really looking forward to the next one. >> Yeah. Thank you so much. All right. >> Bye everyone.