What's new in the Gemini Live API
Read the Portuguese translation →
Author: Google for Developers · Watch on YouTube
Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.
Original language: English
Original video: What’s new in the Gemini Live API · Channel: Google for Developers · 07:10 · Watch on YouTube
Topics: AI agents, AI developer tools, AI models, Multimodal models
Summary
Section titled “Summary”The video introduces updates to the Gemini Live API: asynchronous function calling, proactive audio, client-content injection, and higher background reasoning. Client content can be added without forcing a verbal turn, while proactive audio makes the agent speak only when relevant. Async function calling lets longer-running tools operate in the background while the conversation continues. A demonstration of generating an SVG of a pelican riding a bike illustrates how higher reasoning can work in the background while the model remains responsive.
Key points
Section titled “Key points”- Async function calling lets longer-running tools run in the background while the agent continues the conversation.
- Proactive audio helps the agent speak only when relevant or addressed.
- The send client content feature injects information into a conversation without requiring a verbal response.
- The Gemini Live API now offers higher and background reasoning in the native audio experience.
- A background order lookup demonstrates how the agent can provide updates while a process completes.
- The pelican SVG demonstration compares standard native audio reasoning with a higher-reasoning model.
- The model can keep responding to conversation while working on a complex task in the background.
Chapters
Section titled “Chapters”| Start | Chapter | Summary |
|---|---|---|
| 00:00 | Introducing the Gemini Live API updates | The presenter introduces the Gemini Live API updates, including async function calling, proactive audio, client-content injection, and background reasoning. |
| 01:05 | Client content and proactive audio | The video explains how to inject context without forcing a turn and how proactive audio limits replies to relevant moments. |
| 02:12 | Asynchronous function calling | An order lookup demonstrates how a longer-running tool can work in the background while the agent keeps the user updated. |
| 03:20 | Higher reasoning in native audio | The presenter compares reasoning capabilities by asking the standard and higher-reasoning models to create a pelican-on-a-bike SVG. |
| 05:34 | Background work while staying responsive | The video highlights that the model can handle a complex drawing task in the background and remain responsive to ongoing conversation. |
Quotes
Section titled “Quotes”We’ve introduced async function calling for faster execution, proactive audio so we only speak when relevant — 00:32
By using send client content, you can seamlessly inject information without requiring a verbal response — 01:41
I’m keeping you updated in real time while those background processes complete. — 02:46
The system is designed to manage background tasks like generating graphics while maintaining responsiveness for ongoing conversations. — 06:06
Concepts and entities
Section titled “Concepts and entities”Gemini Live API, Gemini 3.1 Flash Live, send client content, proactive audio, async function calling, native audio, Google for Developers
Full transcript (original language)
Section titled “Full transcript (original language)”00:00 Yo, Gemini. I’m recording a video capturing like all the cool new updates to you, basically. And I want you to, you know, only reply if I address you as Gemini. Um because I’m going to do a quick introduction and then, you know, only reply if I call you Gemini. That all good? >> Got it. I’ll wait until you address me as Gemini. Ready when you are. >> Hello everyone. Today we are very
00:32 excited for tons of amazing updates to the Gemini Life API. So, who better to introduce the updates to you than Gemini Life itself? Hey Gemini, take it away. What is new in the Life API? >> We’ve introduced async function calling for faster execution, proactive audio so we only speak when relevant, the ability to inject context using send client content, and frontier-level background reasoning.
01:05 >> Yes, so some really really exciting features, many of which were highly requested ever since we launched Gemini 3.1 Flash Life. Now, you could see here that I was injecting this client content into the conversation without forcing a turn. So, this is really really useful if you want to backchannel some information. The other really cool feature is proactive audio. So, you could see this here. I basically told Gemini to only reply to us if we address it as Gemini.
01:41 So, you know, we could have a conversation, we can actually prompt the agent to only respond when it is relevant to the the conversation. Uh Gemini, does that sound good? >> Yes, that’s exactly how it works. By using send client content, you can seamlessly inject information without requiring a verbal response, and proactive audio ensures the agent speaks only when necessary. >> Yes, fantastic. So, tons of highly
02:12 requested features. Um another one is async function calling. And now, async function calling is really, really useful if you have some longer-running uh tools that will require time to, you know, come back. And so, they can run in the background while the agent can continue the conversation. So, um Gemini, let’s demonstrate this. Can you check on my order with the order ID 1234 XYZ, please?
02:46 Are you checking for me? >> Yes, I am. I’m keeping you updated in real time while those background processes complete. The results are ready. Your order has been shipped and is estimated to arrive on August 11th, 2026. >> Now, that is perfect timing. Uh really, really fantastic. And another really cool feature is higher reasoning. For the first time ever, we have made higher reasoning and
03:20 background reasoning available in Gemini Live. This brings frontier-level reasoning to the native audio experience. Now, the model I’m showing you here is our standard native audio model, which has very, very good reasoning built in. But, for example, hey Gemini, can you draw an SVG of a pelican riding a bike for me, please?
03:57 >> Here is the SVG annotation of a pelican riding a bike showcasing our new high reasoning capabilities for the live API. >> Yeah, so as you can see, you know, that is not the best of pelican. Um, and so now what I’m going to do is I’m going to switch to our max high reasoning model and you can see the difference there. >> That sounds like a great comparison to highlight the difference in model capabilities for complex tasks.
04:28 >> Okay, fantastic. So now we’re connected with our new high reasoning Gemini live model. So let’s try this out again. Hey Gemini, could you draw an SVG of a pelican riding a bike for me, please? >> Sure, I’ll start working on that right now. >> And while you do that, can we continue the conversation? >> Absolutely. I’m happy to chat while we wait for the drawing to be ready. >> Nice. So what’s new with you?
05:03 >> Not too much. Just here and ready to help out whenever you need anything. >> That’s fantastic news. >> I’m glad to hear that you feel that way. >> All right, tell us about the beautiful pelican that you made. >> I have created an SVG vector illustration of a pelican riding a bicycle complete with a helmet, a flying scarf, and a detailed bike setup.
05:34 >> Now this is a beautiful pelican and this really, really neatly shows the higher reasoning capabilities that are now available within Gemini live and for the first time within native audio. So, you could see that this took some time. You know, obviously this is a pretty important task that is running in the background drawing this beautiful pelican. But at the same time the model stayed responsive. So, was able to put some higher reasoning tasks into the
06:06 background and then come back when it is ready and at the same time stay very reactive and respond to any other questions the user has. Is that accurate Gemini? >> Yes, that’s exactly right. The system is designed to manage background tasks like generating graphics while maintaining responsiveness for ongoing conversations. >> There we go. This is really really exciting. This is really groundbreaking frontier
06:36 level reasoning now and available within Gemini life and the native audio experience. So, try it now now on the Gemini life API and we’re really really excited to see and hear what you will build. Is that right Gemini? >> Yes, that’s absolutely correct. Showcasing the seamless way advanced processing and real-time interaction work together. >> Awesome. Thanks everyone for tuning in and let us know what you’re building. Cheers. >> You’re very welcome and we look forward
07:10 to seeing what everyone creates next.