I Built a Voice Agent You Can Interrupt (Murf Falcon 2, Real Latency Numbers)

Author: Prompt Engineer 48 · Watch on YouTube

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: I Built a Voice Agent You Can Interrupt (Murf Falcon 2, Real Latency Numbers) · Channel: Prompt Engineer 48 · 08:04 · Watch on YouTube

Topics: AI agents, AI content generation, Model inference and optimization, Multimodal models

Summary

The video demonstrates a customer support voice agent built with Whisper for speech recognition, Claude Haiku 4.5 for language processing, and Murf Falcon 2 for speech generation. The agent can look up and change mock airline bookings, stop speaking when interrupted, and respond in different languages. The creator reports a median first-audio latency of 118 milliseconds, while the full voice-to-voice response takes a median of 2.4 seconds. The video also covers troubleshooting speech-recognition delays and incorrect fare information, along with an estimated voice-generation cost of about one and a half cents across the demonstrations.

Key points

The demo uses a scripted caller and a mock airline booking system, with orange on-screen boxes indicating real function calls.
The system combines local Whisper large V3 Turbo, Claude Haiku 4.5 through OpenRouter, and Murf Falcon 2.
Falcon streams audio in chunks so the browser can start playing the first sentence while the language model continues generating.
When the caller speaks over the agent, the browser stops playback and the server cancels the current turn.
The agent detects the language on each turn and uses a matching Falcon voice and locale; the demo switches among English, Spanish, and Hindi.
Across 10 turns, first-audio latency had a median of 118 milliseconds and a range of 113 to 173 milliseconds; a cold connection took around 350 to 450 milliseconds.
Full voice-to-voice latency had a median of 2.4 seconds, with the language model and end-of-speech waiting time making up much of the delay.
A periodic dummy transcription keeps the laptop GPU active and brought speech-to-text latency back to about half a second.
Sending the current booking’s fare rules as the last message helped prevent the agent from incorrectly saying a saver-fare change was free.
The creator estimates about one and a half cents of Falcon voice generation for roughly 1,400 characters spoken in the three calls.

Chapters

Start	Chapter	Summary
00:00	Voice agent demo and booking change	The video introduces an interruptible, multilingual airline support agent and demonstrates it looking up and changing a flex-fare booking.
02:05	How the voice pipeline works	The creator explains the Whisper, Claude Haiku, and Falcon components and how streaming audio reduces perceived wait time.
03:09	Interruptions and language switching	The demo shows the agent stopping when interrupted, then describes language detection and matching Falcon voices and locales.
05:27	Latency measurements and fixes	The creator shares measured audio and full-response latency, then explains fixes for slow transcription and incorrect fare information.
07:33	Cost and verdict	The video estimates voice-generation costs and concludes that Falcon is fast and natural, while the full response loop still takes about two and a half seconds.

Quotes

a customer support voice agent that you can actually interrupt — 00:00

Then, it is one streaming call with the text, a voice ID, the Falcon model, PCM format, and a sample rate. — 02:37

Across the 10 turns in these three calls from India on a warm connection, I measured a median of 118 milliseconds — 05:27

If your agent can talk about money, do not trust the prompt alone. — 06:31

Concepts and entities

Murf Falcon 2, Murf AI, Whisper large V3 Turbo, Claude Haiku 4.5, OpenRouter, Murf Python SDK, Nimbus Air

Full transcript (original language)

00:00 All right. In this video, we will build a customer support voice agent that you can actually interrupt, that switches language in the middle of a call, and that shows its own latency on the screen while it talks. Now, mind you, this video is sponsored by Murf AI, and the agent’s voice is Murf’s new Falcon model. The customer you will hear is also a Falcon voice, a simulated caller that I scripted, so I can repeat the exact same test as many times as I like. Let’s go ahead and see it work. This is a made-up airline called Nimbus Air. First call is a flex ticket. The caller

00:30 gives the booking code, asks for a later flight, and the agent rebooks it. Watch the orange boxes on the screen, because those are the agent actually looking things up, not just talking. >> Hi, my booking code is NB4X2K. Can you check it for me? >> Sure, let me pull that up. You’re booked on flight NA212 from Mumbai to Delhi on October 12th, 2026 at 7:45 a.m., and your booking is confirmed with a flex fare.

01:02 What can I help you with? >> I need to leave later the same day. What flights do you have? >> Let me check what alternatives are available for you. We have NA214 on the same day at 1:30 p.m., or NA216 the next morning at 7:45 a.m. Which one works for you? >> The one at 1:30, please.

01:34 >> Perfect. Let me move you to that flight. You’re all set. You’re now on flight NA214 on October 12th at 1:30 p.m., and there’s no change fee with your flex fare. Is there anything else? >> Perfect. Thank you. That’s all. >> You’re welcome. Have a great flight. >> So, you can see that it looked up the booking, listed the alternatives, and moved the booking to the 1:30 flight. And because it is a flex fare, it told me there is no change fee. Every orange

02:05 box, there is a real function call on a mock booking system. So, how is this built? There are three models in the loop, the ears, the brain, and the mouth. The ears are Whisper, the large V3 Turbo model running locally on this 8-gig GPU laptop. The brain is Claude Haiku 4.5 through OpenRouter, and the mouth is Murf Falcon 2. The trick that makes it feel fast is that nothing waits for anything else. As soon as the language model produces its first sentence, we send that sentence to Falcon, and Falcon streams audio back in

02:37 small chunks that the browser starts playing right away while the model is still writing the rest. On the Falcon side, I’m using the official Murf Python SDK. You create a client with your API key and a region. I picked India since that is closest to me, and there is also a global option that finds the nearest data center for you. Then, it is one streaming call with the text, a voice ID, the Falcon model, PCM format, and a sample rate. And you just loop over the chunks as they arrive. That is really all there is to it. The second call is the one I like. This is a saver fare, so

03:09 it is not refundable, and watch what happens when the caller loses patience. The caller starts talking while the agent is still talking, and it stops, listens, and answers the new question. >> Hello, I’d like a refund on booking QZ81LP. >> Sure, let me pull that up. I found your booking. Flight NA640 from Madrid to London on October 15th at 18:20, confirmed on a saver fare.

03:40 Unfortunately, saver fares aren’t refundable, but I can help you change to a different flight instead if you’d like. >> That’s not fair. Can you explain exactly what my options are. >> I understand your frustration. >> Sorry to cut in. Just tell me the cheapest option. >> The cheapest option is to keep your current booking with no cost. But if you want to fly on a different date or time, a change costs $40.

04:11 >> So you can see the interruption there. The moment the browser hears the caller over the agent, it stops the playback and tells the server to cancel that turn. And notice the agent says, “A change costs $40.” I will come back to that because they did not always say that. Third call, same agent, but now the caller switches language. English first, then Spanish, then Hindi, and at the end the caller asks for a human. >> Hi, can you check booking HT93RD?

04:42 >> Sure, let me pull that up. You’re booked on flight NA108 from Bengaluru to Mumbai on October 9th at 21:00 time, and it’s currently delayed 90 minutes. What can I help you with?

05:27 >> Whisper detects the language of every turn. The language model answers in that language, and I pick a matching Falcon voice and locale for the speech. Falcon has voices in 35 languages, so nothing else has to change. One honest note, Whisper sometimes labels spoken Hindi as Urdu, so when that happens, I run the turn again as Hindi. Now, the part everyone cares about, speed. Murf says Falcon gets the first audio out in under 100 milliseconds. Across the 10 turns in these three calls from India on a warm connection, I measured a median of 118

06:00 milliseconds with a range of 113 to 173. So, it is very close to the claim, not strictly under it, and I think that is worth saying. The very first request on a cold connection took around 350 to 450 milliseconds. So, I open the connection as soon as the call starts. But, the full voice-to-voice time from when you stop talking to when you hear the answer had a median of 2.4 seconds. That includes half a second where the app waits to be sure you have finished talking about 470 milliseconds for speech-to-text, about 850 for the language model’s first words, and then

06:31 Falcon’s 118. So, the voice is the smallest slice. If you want a faster agent, the language model and that silence timer are where you should look. Three things went wrong while building this, and they are the useful part. First, speech-to-text was taking 1.8 seconds on the later turns, but only half a second when I tested the same audio offline. It turned out the laptop GPU was dropping to idle clocks while the agent was talking. So, now I run a tiny dummy transcription every second during a call to keep it awake, and it is back to about half a second. Second,

07:02 the agent invented a discount on a saver fare. It told me the change was free when the policy says $40. Putting the policy in the system prompt was not enough. So, now the service sends the fare rules of the current booking as the very last message on every and across six test runs, it stopped waving the fee. If your agent can talk about money, do not trust the prompt alone. Third, a smaller one, speech-to-text mangled the booking codes until I gave Whisper a short hint with example codes and told the agent to clean up whatever it hears.

07:33 On cost, Murf’s docs list Falcon 2 at about 1 cent per thousand characters. The agent spoke roughly 1,400 characters across these three calls, so that is about a cent and a half of voice. The speech-to-text runs on my own GPU and the language model cost is small as well. So, here’s my verdict. Murf Falcon does what it says on the voice side. It is fast, the voices sound natural, and switching languages is just a parameter. The whole loop is not instant yet. Mine sits around 2 and 1/2 seconds, but that is mostly my own choices, a cloud

08:04 language model, and a safe silence timer. If you’re interested, you can go ahead and try this out on your own system as well and let me know how it goes. I will leave the Murf Falcon link in the description and again, this video was sponsored by Murf AI. I will see you in another one. Check out the next video.