Gemini Live Avatars

Author: Sam Witteveen · Watch on YouTube

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: Gemini Live Avatars · Channel: Sam Witteveen · 09:57 · Watch on YouTube

Topics: AI agents, Multimodal models

Summary

The video demos Google’s Gemini Live Avatar system, which combines real-time audio, video, tool use, and a lip-synced avatar in a single stream. The creator shows how to configure pre-built avatars, voices, personas, system instructions, and tools, then demonstrates conversations in English and Japanese and a debate between two avatars. The service is generally available on Google Cloud, but custom avatars require approval, and the creator notes limitations around flexibility, safety, and likely high pricing. He closes by inviting viewers to consider potential use cases and expressing interest in future open alternatives.

Key points

Gemini Live Avatar produces a rendered, lip-synced face alongside a live conversation.
The system brings together capabilities that previously often required separate speech-to-text, language model, text-to-speech, and avatar services.
It can listen, interpret video or screen content, and call tools in the background; the video mentions support for 97 languages.
The generally available service runs on Google Cloud and offers a library of pre-built avatars.
Custom avatars based on a user’s reference photo or voice require an allow list and verification.
The demo app lets users choose avatars, voices, personas, system instructions, and tools.
The creator demonstrates using an avatar for Japanese practice and for a debate between two characters.
The system can also use a camera so users can show objects and discuss them with an avatar.
The creator notes that the experience has some delay and that flexibility is currently limited.
The compute demands may make the service expensive, while safety controls may restrict avatar use.

Chapters

Start	Chapter	Summary
00:00	Introducing Gemini Live Avatar	The video opens with a racing conversation demo, then explains how Live Avatar combines conversational AI components into a real-time, lip-synced experience.
02:04	Availability and avatar setup	The creator puts the launch in context, explains what general availability includes, and walks through configuring a pre-built avatar.
04:10	Testing conversation and language practice	The demo shows a responsive conversation with Vera, followed by an example of using an avatar to practice ordering coffee in Japanese.
07:18	Multi-avatar conversations and camera use	Two avatars debate pineapple on pizza, and the creator notes that users can also activate the camera to discuss objects.
08:53	Limitations, safety, and outlook	The creator discusses restrictions on custom avatars, potential safety controls and costs, and invites viewers to share possible use cases.

Quotes

every one of those points adds a lot of latency — 01:01

it can listen, it can see what’s on your screen It can call tools in the background. — 01:33

this is actually pretty, responsive, right? — 05:13

these two can actually sort of communicate to each other. — 08:21

the simple fact that this just uses so much compute means that we’re probably not gonna see this get super cheap — 09:24

Concepts and entities

Gemini 3.8 Live, Gemini Live Avatar, Google, Google Cloud, Gemini Search, TTS, Sam Witteveen

Full transcript (original language)

00:00 Welcome. I’m Jennifer, I’m ready to dive into the latest F1 action with you. Hey, Jennifer, who won the latest Grand Prix? George Russell won the latest Grand Prix in Azerbaijan. He had a fantastic race, holding off Max Verstappen to take the win. So where did Kimi Antonelli come? Kimi Antonelli finished in fifth place after a really impressive recovery drive from 16th on the grid Okay, so that was Google’s new live avatars with Gemini 3.8 Live. So I’ve talked about the live models in the past.

00:31 these are the bidirectional models that basically have multimedia going in, being audio and video, and up until recently could just produce text or audio back. Well, Google’s now updated that, so you can actually have a live rendering face that kind of lip syncs what the actual audio is. So up until recently, if you wanted to do any sort of AI agent with a face that could actually hold a conversation, you were generally sort of stitching together four different things. You had some kind of speech-to-text, you had an LLM, you had a TTS

01:01 model creating the voice back, and then there was some sort of avatar service on the end doing lip sync. And the challenge has always been with that kind of thing is that every one of those points adds a lot of latency, and by the time the mouth starts moving, the conversation has often already started to fall apart Now, I won’t say that Google’s got this perfect, but what they’ve done is pretty impressive. They’ve made their Gemini 3.8 live avatar system generally available, And there are actually a few caveats to that, which I’ll mention later on in the video. But this whole thing basically collapses down those four models

01:33 that you had to take care of yourself into a single stream. it can listen, it can see what’s on your screen It can call tools in the background. As you can see in the example I just showed, it was using Gemini Search to basically get the latest results from Saturday’s Grand Prix. And the cool thing here is it can do all of this in 97 languages. so I’ve been playing with this for a while and I built some demos that I’m gonna go through in a second and show you but let’s put this into a little bit of context. So while we still yet to see a Gemini Pro model come out now for, I think it’s about

02:04 seven months this has still been a pretty busy week for Google on the audio side. So in my last video, I talked about the whole new TTS system that they launched with the voice cloning, with those sorts of things. And actually before that, they updated Gemini 3.8 Live and Live Extended Thinking And now they’ve released Live Avatar and like I said in the last video, whatever you think about Google is in the LLM race at the moment, the multimodal and audio stuff has been really strong. They’ve had Omni, Live Translate, Transcribe, the TTS models,

02:36 and now this live avatar. So I guess quickly, what does generally available actually mean here? basically it’s ready for production on Google Cloud. But the sad thing here is that not everything is generally available. You get a library of pre-built avatars. If you want a custom avatar, one made from your own reference photo, your own sort of voice, that kind of thing, that’s behind an allow list and a verification process and even things like the Live extended thinking version of Live is still in private preview. So in this video, I’m just gonna show what you could go and do and build right now

03:08 and show that in my demos, et cetera if you want to see an example of the custom agents, They’ve got a video in there with a demo of making a custom avatar for an avatar with a really bad Australian accent that I think only a bank owned by Crocodile Dundee would actually want to use so with what they’ve actually made generally available in here, You’ve got a bunch of different visual avatars that you can pick from. So here I’ve made an avatar studio, which is basically a front-end app that’s running this and streaming the live avatars through the backend in here.

03:39 So here we can basically just pick any of the avatars that we want to. I can then basically choose a voice, that I’ve got. And they’ve got a set of voices that we can actually pick. we can pick the sort of persona preset that we want. let’s go for a time traveling tour from, twenty one twenty six and then we’ve got system instructions. Now obviously the system instructions You can fully customize yourself. You can get all of that set up. And then you can add in tools if you want. So here I’ve basically got, Google Ground searching, for this. now since we’re talking to someone from the future, that

04:10 actually might not be good. I will leave that off, for now. And then basically what I can do is I can just start it. And then I’ve got the choices once I start Wow, it is so incredible to finally see the quaint 21st century in person. Okay, and you can see there that now, I’m talking to Vera. Vera, how are you doing today? I am doing absolutely wonderful. Thank you. Exploring this era is just pure magic. These are actually a perfect vintage replica of 2026 fashion.

04:42 So you could say I dress the part to blend in perfectly. Okay, so Vera, what I want to show is that we can actually type to you as well. So I’m going to ask you a question in a second, and I only want you to answer the question when you see the typed question Understood. I am ready and waiting for your kite question I’m doing absolutely fantastic. Just soaking in the incredible energy of 2026. Okay, so you can see there that, you can see there that basically she was able to take the type, or voice, and she’s still listening to me, aren’t you, Vera? Exactly.

05:13 I’m processing everything perfectly, no matter how it reaches me. Okay, I’m gonna stop there. So you can get a sense from that, this is actually pretty, responsive, right? Now, there is a delay, and obviously, we’re gonna get the transcript back ever so slightly before the actual voice comes back, which kinda makes sense. but you can see for something where now we can go through and pick, any of the different avatars. you can pick photorealistic avatars, or you can pick, sort of cartoonish avatars in here And so the cool thing here is you can actually make a

05:44 bunch of, fun demos with this. So not only can we do all of this in English, we can actually come in here, and we can pick different languages. So if we wanna have a cafe, and we wanna put it in a different place, we can actually sort of make this a conversational app for teaching us to speak, a different conversation, et cetera. So you can see here, we’ve basically got something where, it’s going to, teach me… let’s try Japanese, right? see how that goes. and what I’m gonna do is try and have a conversation with her.

06:15 Hi there. Can you tell me, how to order a coffee in Japanese? Irasshaimase. Coffee o kudasai to itte kudasai ne. Please say coffee please. kohi is coffee? はい、そうです。コーヒーですね。 Okay, so how do I say “I would like one coffee”? コーヒーを一つください。 Say one coffee please. How do I pronounce it? Can you give me the, corrections for how I should pronounce it? 「コーヒーを 1 つください」と言ってください。

06:45 Repeat after me。「コーヒーを 1 つください」コーヒーをください。ですね。かしこまりました。他に何か注文はありますか？ Okay, so enough of me embarrassing myself, not being able to speak Japanese. But you can see here that we can basically, set this up to, teach us vocabulary, to do a whole bunch of different things. And we could change it so that, we start with, something that’s really beginner and something that then gets right through to something much more advanced in here another thing that you can actually do for fun is that you can

07:18 actually have multiple characters, and have them communicate to each other. So here we’re going to start a debate over, whether pineapple, belongs on pizza or not. And you can see Moderator, esteemed guests, culinary purists often mistake rigidity for tradition, but love and gastronomy thrives on daring unions. To deny the pineapple its rightful throne upon the pizza is to deny the very spirit of romance which seeks the unexpected harmony of sweet and savory.

07:51 Imagine the sun-drenched tropics marrying the rustic warmth of baked dough. It is not just a topping, Carmen, it is poetry on a crust. Quickly it is then. Leo, while I appreciate your poetic vision, culinary harmony requires more than just sweet and savory colliding. From a practical standpoint, the excessive moisture in pineapple compromises the structural integrity of the crust, turning a crisp base into a soggy mess.

08:21 You are right, moderator. There is wisdom in her words that I must respect. Yet, Carmen, true passion often withstands a few. Okay, I’m going to stop it there, but you can get a sense there of the, these two can actually sort of communicate to each other. So, you can have some sort of fun with that. probably I’d go back and actually change the system prompt to make them a little bit more concise, for my demo. And it doesn’t stop there. You can also do things like where, because you can also activate the camera, you can then hold up things and have a conversation about them, with the avatar

08:53 Now, If you want to see a video of me actually explaining the code and going through it a little bit and stuff like that And showing you how to get started, et cetera. I’m gonna put the link for that in the description. That’s gonna be on my second channel. you can actually go through and get this all started yourself okay, so just to finish off, I would say that while this doesn’t have the flexibility that, perhaps we would like at the moment, again, this is Google having to be conservative with how they do things. They don’t want, 500 people uploading pictures of Trump and making a Trump avatar or something like that My guess is there are gonna be quite a number

09:24 of safety features around that kind of avatar system You can imagine also that Google’s going to be pretty strict about the different use cases for this. if you want to try it out, this is in, Google Cloud now. You can actually just come in, And look at the Gemini live avatar. You can pick from the avatars that they’ve actually got set up in here. so you could certainly get a demo of this up for a business or to show people how to use it, et cetera Now, the pricing for this is gonna be pretty pricey, coming out, I think the simple fact that this just uses so much compute means that we’re probably not gonna see this get super cheap in the really near term but

09:57 it’s definitely gonna be interesting to see where this goes, and I’d love to hear from people where you see the best use cases of this are actually gonna be. Let me know in the comments what you think about those And hopefully it won’t be too long before we see an open version of this, perhaps from NVIDIA or from one of the other companies that are actually releasing open models out there as well. Anyway, as always, if you found the video useful, please click like and subscribe, and I will talk to you in the next video. Bye for now.