I Let an AI Voice Approve My Emails. 3 Were Wrong.

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: I Let an AI Voice Approve My Emails. 3 Were Wrong. · Channel: Siraj Raval · 07:50 · Watch on YouTube

Topics: AI agents, Task and workflow automation, Multimodal models

Summary

Siraj adds a spoken interface to his inbox agent and tests approving its replies and calendar actions by voice without reading the log. The voice makes the workflow feel faster and more natural, including in English and Hindi, but its confident delivery leads him to approve incorrect information. At the end, he counts 24 correct approvals and three wrong ones, all of which he says he would have caught by reading. He keeps voice for delivering information but sets a rule: anything that sends, spends, or schedules with another person must be checked as text.

Key points

The agent already reads emails, drafts replies, flags messages needing human attention, and moves calendar items.
Siraj adds a single text-in, audio-out function using Fish Audio’s S2.1 Pro model.
The voice can speak in English and Hindi without switching to a separate model.
For the test, Siraj keeps the log window closed and approves or rejects proposed actions by voice.
The spoken interface feels fast and convenient, but Siraj notices that he stops scrutinizing the agent’s work.
The voice reads a false statement smoothly; the audio is accurate, but the underlying sentence is wrong.
Siraj reports 24 correct approvals and three wrong ones, all of which he says he would have caught by reading.
His rule is to use voice for output, never for approval, and to read actions involving another person as text before they proceed.

Chapters

Start	Chapter	Summary
00:00	The voice-only approval test	Siraj explains that he approved an agent’s work without reading it and sets up a test with the log window closed.
01:01	Adding a voice layer	He describes the limits of a text-based interface and demonstrates adding text-to-audio output with Fish Audio’s S2.1 Pro.
03:09	A confident voice changes the check	Siraj shows how the agent’s spoken delivery made him trust an unchecked claim and begins the working-session test.
04:12	The cost of speed	The voice makes the workflow feel natural, but Siraj catches a false statement and reflects on how confident speech can disguise uncertainty.
05:46	Results and a rule for voice agents	Siraj reveals three wrong approvals out of 27 and decides to keep voice for output while requiring text review for consequential actions.

Quotes

A voice shouldn’t change what it does. It only changes how I check it. — 01:34

Turns out the fastest interface in the world is one where you never have to be sure. — 04:12

24 correct, three wrong. — 05:46

Give your agent a voice for output, never for approval. — 06:48

Voice is a great narrator and a terrible auditor. — 06:48

Concepts and entities

Agent, Fish Audio S2.1 Pro, voice layer, Hindi

Full transcript (original language)

00:00 That was me approving something I never read. That thing that has run my inbox for months. I gave it a voice this morning and by lunch, I’d stopped reading its work. It said one sentence out loud, calm and warm, and sure of itself, and I said, “Yeah, send it.” I didn’t open the log, I didn’t read a line. So, here’s a rule for today. I approve by voice only. The log window stays shut. At the end, I open it and we count what the voice cost me. So, two boxes, I don’t know either number yet, and neither do you. The agent reads my

00:30 inbox. It drafts the replies. It flags the two or three that actually need a human, and it moves things on my calendar without me asking it first. It’s good. It’s genuinely good, and I want you to notice what it looks like when it’s being good. That’s the output. That’s what I’ve been trusting. I skimmed this. I’ve been skimming it for months, and skimming isn’t checking. I know that. It just feels like checking because the screen’s full and it looks like somebody did some work. Nobody’s

01:01 ever built a relationship with the scrollback. It doesn’t matter how smart the model is. If the only way it can reach me is a wall of words at the exact moment my hands are full, I’m still the bottleneck, and that’s not the model’s fault. That’s just a bad way of getting hold of somebody. So, I’m adding one layer. One. Text in, audio out. Nothing else about the agent changes. Same model, same instructions, same permissions, same everything. A voice shouldn’t change what it does. It only

01:34 changes how I check it. Remember that one because that’s the bit I turned out to be wrong about. So, 90 seconds on how I built the voice, then I close the log window and we start counting. So, the agent works. It reads my inbox, drafts the replies, flags the two that need me. The problem is that it talks like a log file. I want to hear it. For the voice layer, I’m using Fish Audio’s S2.1 Pro model. One reason, it holds a voice across languages. [music] My audience is half India an agent that can switch between English and Hindi without

02:05 sounding like two different robots. The whole integration is one function, text in, audio out. Here’s the call and here’s a voice ID, here’s the request. That’s the agent reading me the flagged email. Same voice, Hindi. No retraining, no second model. Two more things I wired into the same function. This is the morning brief. It reads me what happened overnight while I’m making coffee in the same voice. And this one narrates a bedtime story. Same API call, different text. That’s the [music] point. Once the voice layer is one function, every text

02:37 your agent already produces becomes something you can listen to. If you ever want to build the S2.1 Pro developer API, it’s free through the end of November subject to the fair use policy. So, you can start testing it on your own agent for free. The link is the first line of the description. Now it can talk. Back to the build. So, that’s the build, one sitting, one function. Honestly, the code is the boring bit here. What I didn’t expect was that what it’s actually like to live with the thing. I gave it a name on day one. I’m

03:09 not telling you what because you’d judge me and you’d be right. I started saying please, then I apologized to it for interrupting out loud to software. And that’s funny right up until it isn’t. Look at that. That’s me the same day I gave it a voice. Listen to what it did. >> Elfie I collect came back on Salty. She can’t take the X post. She needs a YouTube placement and nothing is signed with the brand yet. >> It didn’t check anything. It sounded sure, so I was sure. The information was identical, same agent, same draft, same

03:40 decision. The only thing that changed was that it arrived in a voice and my whole standard for checking it just quietly went away. So, let’s find out how bad that is properly. One working session. The agent reads me what needs me out loud. It proposes the reply out loud. It proposes the calendar move out loud. I answer yes or no by voice fast, the way I would if a person were sitting here, and that window stays closed until the end of this video. I have to tell you for a long stretch of that session,

04:12 this is the best my setup has ever felt. I’m not at the desk. It’s calling my day at me. It moves one thing. It kills another. I’m saying yes and no with my hands full, and it’s fast. Turns out the fastest interface [music] in the world is one where you never have to be sure. Hold that feeling because a few approvals from here, it says something completely false in exactly that tone. Then this comes in. It’s a threat that’s genuinely better handled in Hindi, and I want to hear it the way I’d actually say it. That’s the moment it stopped feeling like a demo to me. Not because it’s

04:44 impressive, although it is, because it’s the first time the thing sounded like it belonged in my house in both of the languages that are actually spoken in it. One voice, not an English robot and a Hindi robot taking turns. And I want to be honest about what that did to me because it did not make me more careful. Later in the session, this happened. That’s wrong, flatly wrong, and I only caught it because I happen to know the answer. Now, I want to be precise about what just failed because it’s easy to blame the wrong thing. The voice did its

05:16 job. It read that sentence perfectly. The sentence was just wrong. That’s not the speech model’s fault. It never had an opinion about whether any of it was true. But here’s the part that got me. A log file looks uncertain when it’s uncertain. You can see the hedging. You can watch it argue with itself. A voice never does that. A voice is smooth all the way through whether it’s right or not. And that’s what got me because I’d spent the whole day treating how sure it sounded as [music] proof it was right.

05:46 It isn’t proof. it’s a tone of voice. All right, let’s count. 24 correct, three wrong. [music] So, there it is. I said those two boxes out loud this morning before I knew what was going in them, and I said I’d publish whatever they said. And every single one of those wrong ones was a thing I would have caught in the moment I read it. So, honest verdict, my speed went up by a lot, and my scrutiny went down by about the same amount, which I didn’t clock at all until those counters were sitting on the screen. I kept reading that as two

06:18 results, a good one and a bad one, but it’s not two. It’s the same number twice. If something lets you decide faster, you’re getting less time to be suspicious. That’s just what faster means. It never tricked me. It took away the friction, and it turns out the friction was doing my thinking for me. So, two things I’m keeping, the voice for output, for reading things to me, for finding me where when I’m nowhere near the desk, that part is genuinely better than a screen, and I’m not giving it back. And a hard rule, which is that

06:48 anything that sends, spends, or schedules with another human gets read as text before it goes by me, [music] with my eyes. And I’m keeping it on a short leash because the leash is the only part of this that I control. If you build one of these, and a lot of you are about to, this is the part I take with you. [music] Give your agent a voice for output, never for approval. Voice is a great narrator and a terrible auditor. That’s really it. I don’t have a longer version. And to be clear about where the

07:19 fault line is, because I got this wrong myself for an entire working session, that’s not a statement about a speech model. The audio was flawless. [music] It’s a statement about me and about what a calm, confident delivery does to a human being’s willingness [music] to check. Every time you go faster, you’re paying for it somewhere. And when it’s a voice, you you hear yourself paying. Last thing, tell me in the comments what’s the one thing you’d never let a voice approve. Links in the first line of the description if you want to build

07:50 a voice layer yourself. Go make yours talk, then go read its work.