Act, Confirm, or Stop? Smarter behavior for AI assistants, wearables & robots — Amit Desai

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: Act, Confirm, or Stop? Smarter behavior for AI assistants, wearables & robots — Amit Desai · Channel: AI Engineer · 20:21 · Watch on YouTube

Topics: AI agents, Multimodal models, Robotics and autonomous systems

Summary

Amit Desai argues that improving an AI assistant’s accuracy is not the only way to improve user satisfaction: systems can also make smarter decisions about how to respond when uncertain. Using a smart-speaker example, he compares acting, stopping, and confirming, and shows how assigning costs to different user outcomes can help choose response thresholds that minimize overall user effort. The approach can improve the experience without changing the underlying accuracy and can be adapted to other interfaces, including TVs and embodied AI. Desai concludes that minimizing the “ouch” of mistakes and unnecessary interaction is important as AI systems take more consequential actions.

Key points

Voice interfaces are natural and powerful, but recognition errors can cause increasingly serious consequences as AI systems take digital or physical actions.
User satisfaction can be improved through two distinct levers: greater accuracy and smarter system decisions under uncertainty.
A system can choose to act or stop, rather than always executing its best hypothesis.
The decision to stop should account for the unequal user costs of a wrong action and a request to repeat.
Desai’s “outcome user cost heuristic,” or “ouch,” models those costs and supports choosing a threshold that minimizes them.
Adding a confirmation behavior creates three response options: stop, confirm, or act; their thresholds can be optimized using the same cost-based approach.
The example improves user experience without changing the assistant’s accuracy.
The method can transfer to other interfaces, but cost assignments should reflect their modalities and consequences.
For embodied AI and other action-taking systems, conversational behavior under uncertainty is increasingly important to acceptable user experience.

Chapters

Start	Chapter	Summary
00:01	Voice AI’s promise and the need for smarter behavior	Desai introduces his background and explains why the growing consequences of voice errors call for an approach beyond accuracy improvements.
03:19	Accuracy, confidence, and the stop option	Using a smart-speaker example, he defines accuracy and introduces stopping when confidence is below a threshold.
07:01	Quantifying user costs and minimizing “ouch”	He explains why wrong actions and stops have different costs, then uses a cost function to select a better threshold.
11:55	Adding confirmation as a third behavior	Desai adds confirmation and describes optimizing the thresholds separating stop, confirm, and act.
15:44	Applying the approach across interfaces	He summarizes the gains possible without changing accuracy and discusses adapting cost values for TVs and other AI surfaces.
18:27	Conclusion: minimize the user’s “ouch”	Desai connects cost-aware behavior to embodied AI and argues that better interaction under uncertainty is essential to user experience.

Quotes

there are two ways to improve customer or user satisfaction of a voice AI assistant — 02:15

we could introduce at least one system behavior to stop — 05:24

this whole thing is called an outcome user cost heruristic — 09:45

The system is not any smarter in that sense. — 11:23

it would be to minimize the ouch of the experience. — 19:36

Concepts and entities

Amit Desai, Voice AI, Outcome User Cost Heuristic (OUCH), decision model, embodied AI, Alexa

Full transcript (original language)

00:01 [music] Hi everyone. How’s it going? Hey Patricia, how are you? >> Uh so last presentation of the day, so let’s make it count. Um, all right. Let’s, uh, let me start with a little bit of background on myself. And, um, my background, I’m a voice subject matter expert. I’ve been working

00:35 in voice AI for a long time across different surfaces, devices, and um, both at Alexa, at at Roku, at my own startups, you know, in the app store. And my perspective is a little different from a lot of other voice AI practitioners. I think it’s a combination of um a deep um voice user interface expertise and intu intuition mixed in with new technical approaches

01:09 uh that I think can produce really magical experiences. So I think it’s both sides and I think that’s especially true in this new area that we’re in with frontier tech where the human interface is basically being redefined. So let me start with uh I’ll just blast through the first couple of slides then get to the premise. I think everybody knows that voice has incredible potential. There’s the power of voice I think across everywhere. It’s the most natural interface. Humans love talking. And uh the problem is the other half is the

01:43 pain of voice. So it’s the power and the pain. Voice is errorprone. And I think those errors are going to continue for a while. And I think the cost or consequence of those errors is going to grow, especially as we go fromational AI bots to embodied AI where rather than just giving answers that might be erroneous, we’re going to have AI systems take physical actions or digital actions where, you know, if the robot throws

02:15 your watch out with the trash, it’s a lot worse than playing the wrong song. So I do think that a new approach is definitely needed and here’s the TLDDR of the premise we’re going to walk through today. Um there are two ways to improve customer or user satisfaction of a voice AI assistant and that is by increasing accuracy which people know about I mean technically accuracy and the other is a different knob that we have that we are not using adequately

02:48 and I’ll call that a system decision which we will define which is orthogonal which is different from accuracy and I believe This approach which I have used in several different environments and seen some success I think is a promising area that we should consider developing. Um let me walk through this with a simple smart speaker example and we’ll go step by step with this approach but

03:19 it is a scalable approach that I think uh can apply across different surfaces and devices. So let’s get started. So suppose we all you know are making a smart speaker coincidentally called uh Alexa and Alexa is very simple. It just allows you to you know ask for music and it’ll play a song and of course it will play either the song you wanted or a different song. So it’ll be right or it’ll be wrong. This isn’t that different from what you’ve seen out

03:51 there. Um now let’s to first talk about accuracy. Accuracy. Let’s say we define it as we you know take a thousand spoken requests. We observe the input and the output. We label it and we look at this. This is the map of a thousand points and 79% of the time 790 dots here were actually the correct song. This is let’s say human annotated 20% 21% wrong song. So that’s the accuracy. Now, like I

04:22 said, knob one is to spend a lot of time working on improving the accuracy, you know, um, percentage point by percentage point at any layer in the stack. There’s, if it’s a cascaded system, you know, there’s a perhaps a wakeword layer and a speech ASR layer and a NLU layer which might have intent classification, entity extraction, a lot of different layers, VAD, etc. And any of those can contribute to errors. So we spent time we might be able to reduce that 210 to a

04:52 smaller number that is I think a known area that we’re tackling but I think knob 2 which is what I was talking about is what we’ll go through here which is keeping the accuracy exactly the same. So 79% what could we do in conditions of uncertainty to improve user satisfaction apparent and I I think we can do a lot. So let’s start first with the original system is just acting like I said user says something system plays a song it’s either the right song or the wrong song

05:24 immediately I think just common sense tells us that we could introduce at least one system behavior to stop or rather to reject the hypothesis and do nothing. So uh there is now one more option to decide the system may decide and say sorry I didn’t get that or sorry could you repeat that? Uh the challenge of course is how how when do we decide to stop and I mean quantitatively. Um here’s one approach to kind of visualizing this because if we don’t

05:55 we’ll just take probably some swag like some guesstimate and I’ll prove that if we just took a guesstimate we would end up with a worse situation than a more rigorous approach. So let’s just assume I took those thousand data points and like I said they’ve been annotated and we assign a confidence score a single confidence score to the hypothesis that was generated by the system you know between zero and one and let’s say it’s reasonably calibrated. This is a simplification of if it’s a cascaded system there are multiple layers and multiple you know confidence scores but let’s just assume that for now. Whoops.

06:28 So we’re going to have 790 points 200 that are correct 210 wrong. Each one has a confidence score and we’re going to plot it, you know, plot the distributions. Uh on the x-axis, I’ve just converted from 0ero to one to percentages. And the question is how do we choose a threshold t such that whatever that percentage is um to the left of it meaning if when the system um forms a hypothesis if the confidence score c is less than that t stop and say

07:01 sorry otherwise play question is how do we choose a t so far everything I’m saying is fairly common sensical but this is where um intuition will fail us we might say something like okay I don’t know let’s do 65%. It seems you know gut feeling like okay it’s kind of confident that’s probably when we should speak. Um now here’s where we start coming out with some sophistication. Any tea we choose is producing bad outcomes. Bad in the in two fields. One

07:33 is obviously on the left side anytime you stop it’s bad. The user doesn’t want it to stop. He wants to they want to hear their song. The other bad is if you do play a wrong song, of course that’s bad as well. So these are two two kinds of bad outcomes. But here’s the important part. Now I’ve like elaborated on the um tree diagram on the right hand side. The bad outcomes are not equally bad. They’re not the same thing from a user perspective. And obviously let’s

08:05 let’s think about it. If the wrong song plays, you said play kiss and it starts playing kiss by Chris Brown instead of the one by Prince. That’s going to be um the highest user cost. Now I’m defining user cost from the user’s perspective. First I have to like hear music and realize that is not Prince. Then I have to shout over my Alexa and um you know get it to stop and then I have to re-request. All of that is a lot of effort. that is definitely a worse

08:37 outcome than the system stopping and saying sorry I didn’t understand that however we should go further and try to quantify that relative badness and there many ways to do it and I think this is an area to be explored for now let’s just consider this a heristic of if that outcome happens how many more seconds additional seconds will it take for the user to get back to success which is to play the song they wanted kiss by Prince and I’m I just put down some numbers here. Let’s say in the case of a bad song, it’s 10 seconds if you add up all

09:10 the things I got to do. And if it’s a I didn’t understand you, it’s 4 seconds because that’s how long it would take you to respe and and the extra latency. And now here’s where we can start utilizing that. If we go back to our distribution curve on trying to find out where is T. Now we’ve basically turned this into a problem of minimizing a cost function. It’s a user cost function. It is the number of bad acts wherever that whatever the t causes times 10 because that was a unit cost we gave plus the number of stops times four because

09:45 that’s the the unit cost we gave. By the way, one thing I should have elaborated because I work in voice and we like language and we like puns. So this whole thing is called an outcome user cost heruristic. So that spells the word ouch and that is some expression of pain. Yes, we are you know language nerds. So these kinds of things amuse us. Um so now let’s consider that is the cost function is to minimize the ouch. And now um that let’s see if uh I’m going to

10:16 bring up a tool. Let’s see if this works. Where I have actually gotten or with one of my coding assistants gotten uh an interactive um graph where we have actually plotted those thousand points and as we vary the threshold t you can see that the total user cost here which is that function of you know x * y + a * b actually changes. So let’s in the very beginning when we said the system was just playing

10:52 the the cost across those thousand points was 2100 or divided by a,000 is 2.1 ouch points per turn. Then we said okay let’s insert a stop behavior and let’s like wing it and say 65%. That’s when I want the threshold. If we brought this up to 65 yeah that’s better. Now it’s 1904 or 1.9 per turn, but it’s not optimal. As it turns out, if we do actually um ask for the AI to solve the

11:23 uh the problem across this curve, it turns out 43%. So I’ll drag it now to 43 is in fact the optimal optimal point of t. This minimizes the cost function. You can see it’s the lowest point on this graph down here to 1 27. So effectively we haven’t changed the accuracy at all. The system is not any smarter in that sense. But with some clever system behavior, conversational

11:55 behavior is what we’d call it and some optimization and a cost function called ouch. Um we have from the user’s perspective produced a more satisfactory assistant. And this is not a trivial you know accomplishment. Okay. Now, let me go back to this. [clears throat] Let me see if I can get this. Oh, great. Okay, let’s continue this. Let’s continue this with by now adding one more behavior. Let’s call it the confirm behavior. So, there was play obviously, then stop,

12:26 confirm. Confirm is basically the system after you said something saying uh kiss play kiss by Prince or maybe play kiss by Chris Brown. And uh you know the user can either confirm like affirm it or they can correct it. It is a different kind of behavior and again this is kind of how humans behave. Um that’s obviously the inspiration. Now if we go back to our problem of optimization, we have a third obviously um option which is to confirm. And so this would

12:58 translate to two thresholds um two thresholds which are separating the distribution into three spaces of stop, confirm and uh act. And the question is now where are these T’s? and we have now given up on guesstimating because we know it doesn’t work. So we’re going to be a lot smarter and go back to the concept of user outcome cost and then you know use it go look for some optimization in that graph. So let’s uh define what are the what are

13:32 all the possible bad outcomes that t1 and t2 um make for. So good you can see my cursor. So uh of course any stops are still bad. Then in the middle are confirmations. Confirmations are bad because they slow the user down. There is a confirmation outcome called confirm yes where they just affirmed it by saying yeah or no where they had to correct it. And going back to our formula these outcomes are not equally bad. And in fact, nobody will, I think,

14:06 argue here from a user’s perspective. Affirming, just saying yes is obviously less painful than saying no and then having to restate whatever it is that you wanted in the first place. So now we I’ve assigned values of two or six. And again, I said it was a heristic. This would be roughly the amount of time it would take for the extra for the user to get to the song they want. Saying listening and then saying yes is like two seconds. Um and then now uh we restate the cost function for this you know added behavior as this number

14:40 of you know bad type one times unit cost bad type plus bad type two times unit cost etc. And now we try to minimize this user cost function and minimize the ouch. Yes, I’m going to keep doing that pun. Um let’s go back. So this is now the interactive graph but with um the cost values the unit costs here 10264 and uh you know we’re just going to ask

15:13 the AI to tell us here’s the heat map because it’s now two dimensions saying that the optimal values are 41 for the the T1 and 49 for the T2 and if we employed that then we would go to 1464. Uh, by the way, whatever numbers I put in here, like let’s say I thought wrong act was 20. It’s really irritating and painful and takes way longer to actually correct it when you hear a wrong song.

15:44 That would change you know all these numbers uh and the optim optimal point. So again it is about how what is the relative badness of these outcomes also of course the distribution curves naturally. Uh let’s go back here. Okay. So, um I’m gonna speed up a little bit. Uh let’s go back here. Presentation mode. Okay. So, what have we shown that if we did the super naive approach, it’s 2.1 act and stop 1.9 then

16:19 1.27 then 1.26. We are able to bring this with every added layer of sophistication, adding more behaviors, being smart about outcome, uh, user cost and optimizing. Um, we have made a tremendous difference without changing the accuracy at all. Um, this was a super simplified example. In real systems, you’re not going to have obviously some offline decision threshold or two. It’s going to be a real time, you know, learned decision model. But the principle is the same. And I believe this is uh scalable across

16:54 all voice AI surfaces. Obviously this is a smart speaker but if we go across any of these surfaces you will find the equivalence. If we um we will find the analogies with some differences but the spirit and the I think the the gain will be similar. So just for example in the TV AI assistant space if you employ it here it’s going to you’re going to have the same thing when users express

17:24 intents like on TV it’s you know open a channel that’s one of the most common obviously um requests on a TV voice assistant same thing you’re going to find you’ll have exactly the same approach but the difference will be maybe in the the assignments of the user outcomes because the UI and the modalities are different when you have a TV you have a multimodal interface where choices can be shown. So instead of you know asking did you mean ABC you know uh

17:55 news live by speech that you will the system would display choices and not just one it show ABC News live this that would be the confirm step and if it’s visual and you can use your remote control to select something it’s less pain so you would change some of these values or if in fact launching the channel would kick you out of your current state then it would go in the other direction than cost of you know a bad act would go much higher. So it’s the same concept but in this new modalities

18:27 um variables can change, values can change, arguments can change but the premise still holds and you can improve from the user’s perspective because we’re all about you know making humans happy. Um you can make them happier and this as I said in conclusion can be applied across all surfaces. I did say at the very beginning, just to recap for us, that voice is great when it works, bad when it doesn’t. And as we get into embodied AI, where these AI assistants

19:01 are taking actions, physical or even digital, like making a phone call or sending an email, it is getting more and more difficult just to rely on accuracy to improve user satisfaction. I believe there’s a whole knob the second knob called smarter conversational behavior under uncertainty and um if we actually exploit that we can uh very much help these AI systems reach a acceptable user experience otherwise I think this will continue to

19:36 be a bottleneck like a lot of things will get better but if the voice interface as experienced by user does not improve it is going to be a a a choke point. And um if you just remember one word or two words from this whole um presentation, it would be to minimize the ouch of the experience. Um so thank you. I’ll stick around for questions if you guys got any. Thanks a lot. [applause]

20:21 >> [music]