Gemini 4 Argon: What’s new and what’s next

Author: Google for Developers · Watch on YouTube

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: Gemini 4 Argon: What’s new and what’s next · Channel: Google for Developers · 29:46 · Watch on YouTube

Topics: AI agents, Prompt and context engineering, Software engineering with AI, AI models, AI safety

Summary

Google for Developers discusses Gemini 4 Argon’s launch across the Gemini app, AI Studio, Gemini API on Cloud, and Antigravity, along with its testing and safety work. The speakers describe internal use cases including code migration, software and hardware debugging, writing assistance, and research, while emphasizing the need to verify outputs and keep human judgment involved. They highlight the model’s ability to generate up to one million output tokens in a single step for demanding tasks, and discuss how persistence and transparency are balanced in its behavior. The episode closes by framing Argon as part of a faster model development cycle that will continue to improve through user feedback.

Key points

Gemini 4 Argon is available in the Gemini app, AI Studio, the Gemini API on Cloud, and Antigravity.
The team tested the model with cyber defenders, enterprises, customers, internal users, and researchers, while investing in safety guardrails and monitoring.
Google employees use Argon for tasks such as writing assistance, code migration, Chromium work, performance optimization, and debugging hardware issues.
The speakers encourage users to scrutinize model outputs, ask follow-up questions, and iterate rather than treating a first response as definitive.
The team describes a faster development cadence in which internal use of models helps inform subsequent versions.
Argon can produce up to one million output tokens in a single step, which the speakers say can help with challenging reasoning and math problems.
The model’s behavior is tuned to balance persistence with clarification, and transparency with concision.
The team gathers feedback from Googlers and runs experiments to assess model behavior and user experience.

Chapters

Start	Chapter	Summary

Quotes

So it should feel more concise, more precise. — 05:45

you need to have skepticism whether it’s an AI presenting you information or a person. — 14:04

read the output yourself, make sure that the output makes sense to you — 16:42

the model has the ability to really go to one million tokens if it can. — 24:01

we want it to be able to clarify ambiguity. — 27:40

Concepts and entities

Gemini 4 Argon, Gemini app, AI Studio, Gemini API, Antigravity, Wiz, Google DeepMind, Fortran, Rust, um milhão de tokens de saída, Logan Kilpatrick, Josh, Melvin, Sergey, Tulsee

Full transcript (original language)

00:00 We’ve seen performance optimizations across the board, across use cases. Some folks on the Antigravity team have been using the model to migrate from Fortran to Rust. So, code migrations. - Wait, who’s using Fortran in the— - No, they’ve been taking Fortran code bases. - External Fortran code bases. - External code bases. It’s like a museum kind of thing. - Yeah, it’s like the world runs on Fortran. But there are still code bases in Fortran externally. - Who here has ever— I have written Fortran. - I have not. - Oh, you have? Nice. Okay.

00:31 Anybody used punch cards, paper? No. No? I mean, I’ve seen them, I haven’t really used them, to be honest. Hey, everyone. How’s it going? Welcome back to Release Notes. My name’s Logan Kilpatrick. I’m on the Google DeepMind team. Today we’re here talking about Gemini 4 Argon. I’m super excited. We’ve got Josh, Melvin, Sergey, Tulsee, the entire crew spanning product, model, research, everything in between. So, maybe, Josh, maybe you kick us off with the news.

01:02 Gemini 4 Argon available, today, for people to use. - That’s right. It’s going to be in the Gemini app, in AI Studio, the Gemini API on cloud, as well as Antigravity. And we can’t wait for you to try it. It’s got a bunch of leaps. We can unpack it here today. - I love it. Yeah, I think we announced the model benchmark charts. We handed it off to a bunch of folks in the Fairwind program, cyber defenders. And maybe actually, it’s worth just starting with what’s been going on. We showed a bunch of, we showed a nice, beautiful benchmark chart,

01:33 two weeks ago roughly. And between now and then, all this stuff, I know we can talk about it from all different angles, product testing, early access research, et cetera? - Yeah, it’s been great. So I think, like you said, we announced it about two weeks ago. And since then, we’ve been doing a few things that have been really great. One is just continuing to test the model. So we’ve been giving it to a set of cyber defenders and have actually been working with Wiz, which is an internal cybersecurity company, through their Scan For Good program to actually already start identifying and patching vulnerabilities

02:04 across a bunch of critical systems. We’ve actually started also testing it with enterprises and customers, so slowly rolling out to start getting some feedback. We’ve been testing it, of course, internally a ton, and also with some of our researchers who are focusing on different domains, right? So quantum, robotics, science, and that’s been really interesting. And then lastly, but really critically, we’ve been really investing in the safety story. So that doesn’t just include our relationship with the US government where we submitted the model for testing, but we’ve also been improving our own guardrails, right? Really making sure we have the right monitoring setup,

02:36 the right internal guardrails within the model, external guardrails around the model setup. So that we’re actually very comfortable releasing tomorrow or today or yesterday whenever this podcast drops. - Whenever this podcast drops. Sergey, I feel like the internal usage of the models to me is one of the biggest— feels like one of the biggest differences in how we’ve gone about thinking about what is a great model to bring to the world. And we have 100,000 amazing engineers who are using the model. Varun quoted this earlier when we were catching up with him, 100,000 people internally testing, which I feel like is

03:08 probably more than we’ve ever had for any model released before in the past. And so I’m curious how, I feel like you’re at the helm of a lot of this internal flywheel and the testing that’s happening. - Yeah, I’m very excited about the cadence that we have nowadays. And if you look at even before Gemini 4, when we had Flash 3.6, 3.7, 3.8, we’re starting to get onto that RSI treadmill. And now the, you know, our researchers

03:39 and the model work hand in hand to make the next version of the model. Actually, I just popped in here randomly because Melvin and I were talking about the next version of the model. And, probably by the time folks are viewing this, internally we’re already using a couple models ahead. So it’s just very exciting to see the progress and the cadence. And I expect it’ll only continue to accelerate. - Yeah. As the acceleration continues to happen, almost like the bar for where

04:09 we bring the model out, or what the launch bar actually is, I feel like continues to change and evolve because it feels dynamic. And so I’m curious, from a research perspective, also Tulsee, maybe from the model product perspective, how, yeah, how are we dealing with that shifting bar and like what is good enough to bring to the world? - One thing about Argon that, I feel really good about is we have extensively tested the model for, more than three, four weeks now internally and all of the signs tell us that it’s good. But what I’m excited about is getting it out to a broader set of use cases

04:42 and a broader set of users across Gemini app, Antigravity, and cloud. So when we’re in this fast cadence where we’re building the next one, we can actually iterate on that feedback. Right now we’re getting bits and pieces but we want more and more people to use it. And we’re on this fast three to four week cycle. Every couple of weeks we have candidates every three weeks we can graduate some promising ones. But if we can inject that feedback back into how we design the models, that’s going to get us even better. - Yeah. Well, something that’s interesting is I’m on a Varun quoting

05:15 chain right now, and he mentioned earlier that one of the most— other than engineers inside of Google, I feel like one of the other top personas is the cloud sales team using Argon internally. And so it’s like this whole knowledge. It’s not just great for coding, great for all those use cases, great for science, but also, great for knowledge work. And so I feel like in the Gemini app, a bunch of updates landing. Josh, I don’t know if you want to talk through any of those. - Yeah. So, well, one we focused a lot on that hopefully everyone will feel is just writing quality.

05:45 It has a house style that feels unique. And so part of that started, we’ve been tracking feedback on this and there’s AI slop everywhere, though, load-bearing phrases, to be honest, listification, all this stuff. And so we made a real concerted effort to really try to just stamp that out. So it should feel more concise, more precise. We’ve worked with a bunch of professional writers, and they’ve just been marking up the model output. And we turn those into goldens, and we climb on that.

06:16 And so there’s a lot of thought that’s gone into that. So yeah, whether you’re on the sales side, the legal side, the finance side, even writing code, obviously, it’s great at. But hopefully you’ll feel that it doesn’t feel or sound like another model. It has its own tone. So that’s been a big focus for the Gemini app and beyond. - Yeah. What my really quick personal question is, I don’t use AI for any writing. And so like, do you think this– I don’t know if you felt this for yourself because you do a lot of writing actually internally, externally. And so has it crossed that chasm where you’re willing to start

06:46 letting it be more in the driver’s seat? Because I’m like 100% artisanal. I type or write every word. I want my words coming out, also because of all the right to think stuff. - Yeah. - But yeah, I still think— - I still write, so, I haven’t given up writing, but it’s an incredible copy editor. - Got it. - And so usually I’ll write something— actually, I’ll dictate something when I’m walking somewhere or in the car, and I’ll throw it over to Argon and say sharpen this. What are the three weakest parts of this? What are three ideas I didn’t think of, three ideas that might confuse people.

07:17 It’s exceptional at this. - Yeah, and I actually, like, I’ve been having wrist issues, and so I actually will just ramble at Antigravity and then have it turn that into writing. And I found that, actually usually that’s pretty hard to do because it comes out very, dense, clunky, very AI sloppy, and, then it doesn’t actually feel that good. But with Argon, I’ve been able actually to do that a lot more and actually find the artifact reasonable, which then I can then edit. But at least the concept is still from my thinking, but it’s more from my verbal input, which I think is actually pretty cool.

07:50 - I love that. Yeah, Any other good internal use cases or examples that folks have seen that have really stuck out as Googlers have used this model at scale? I’m curious. - I think this is probably the first time Sergey can attest to this because he’s been pushing us to do that, is having agents in the model building process in a more substantial way. So, we actually have agents monitoring the hero runs. We have agents giving feedback on misconfigurations. We have agents looking at reward scales or finding bugs, patching bugs,

08:21 agents looking at data and duplicates and stuff like that. I think that’s exciting, and we’re only going to get better at that kind of thing. - Did you see the data center game that was going around? - No. - Did a simulation of our data centers. And as they’re running these training jobs and even down to, like, people having to go and swap GPUs and all those things. I don’t know if it just went around the chat. It’s amazing. I mean, the kinds of things you can do now with just a prompt and you just come back and there it is.

08:54 - We put out a call to Googlers to ask them, like, what are the exciting things that they’re doing with Argon and what are some of the use cases? And we’ve been compiling a lot of that as just to better understand how the model is working. And there’s been a bunch, like the Chromium team has been using the model pretty aggressively to address some of their work. We’ve seen Gemma optimizations. We’ve seen performance optimizations across the board, across use cases. Some folks on the Antigravity team have been using the model to migrate from Fortran to Rust. So code migrations.

09:25 - Wait, who’s using Fortran? - No, they’ve been taking Fortran code bases. - External Fortran code bases. - It’s like a museum thing. - Yeah, it’s like— the world runs on Fortran still. There’s still code bases in Fortran externally, and so they’ve been trying to do just like— okay. - Who here has ever— I have written Fortran. - I have not. - Oh, you have? Nice. Okay. Anybody used punch cards? Paper? - No. - No? - No. - I mean, I’ve seen them. I haven’t written them, to be honest. - That counts. - I’ve done BASIC, COBOL.

09:57 - I’ve done BASIC. - Oh, yeah. No, BASIC. I’ve never done COBOL. Wow. - You got to get back into it. There’s a big market. COBOL developers. - For migrations. - Yeah, for migrations. - I think, okay, one other example, actually, Sergey might know this also, is we’ve had these hardware issues during our hero runs where we would get some bad state in the machines. And we’ve been trying to get to the bottom of it for a long time. We’ve learned to live with it. And then one of the engineers sent the problem to Argon

10:28 and it found an actual issue in the hardware. We took it back to the hardware team in Sunnyvale. They’re like, are you bringing us AI slop? We don’t want to listen to you. But we actually, went, sat there with them and they actually went back to the provider and found corrupted HPM. - Wow. - So this is pretty good. Now we’re able to identify all of the machines that come from that vendor and when we can outcast them from the fleet. - Melvin, you didn’t submit this into what are the cool examples. - Sorry. - I should have. - I should have. - Right now, live.

10:58 - I should have. - You know, funny story. The first twenty or so servers we bought for Google, it must have been ’ninety-eight, ’ninety-nine maybe. But I just couldn’t get the operating system to install on these servers. We’d had like two machines before and we scaled up to twenty and just none of the new ones were working. And I debugged and debugged and debugged and eventually I ran a memory tester and I realized there was an incompatibility

11:29 between the memory and like the motherboard or whatever. They’d actually given us fancier memory for free, but it didn’t work. But I had spent days figuring that out, so thank you, Argon. I guess in today’s world I wouldn’t have had to do that. - Were those in the Lego server racks back–that was before. - No, okay, the Lego, which was in fact an off-brand Duplo knockoff called Mega Bloks. Yes, my son loves those. No, that was from Stanford days.

11:59 Yeah, they’re cheap, but they don’t have the same tolerances as Legos, but it worked for a hard drive case. Anyway, no, that was still at Stanford. And then once we became a company, we went and we bought from KingStar Computers, I think. It wasn’t entire— it was their fault, actually. It was KingStar’s fault. But whatever. There was an incompatibility between the memory they’d selected and the motherboard. Memory. - Yeah. - And to be fair, I think they used a higher end of memory than the motherboard should have had.

12:30 But it just would periodically corrupt. It’s one of those things. They’ll run for a couple hours fine and then boom. Yeah. So thirty years later, same thing. - And then the ECC does not catch this corruption. So that’s why we couldn’t find it or we couldn’t root cause it. - Melvin, this is an interesting example and I feel like we’ll hear a lot more of these, but like this, the models, more people being able to push the models now as Argon rolls out to more folks and like it can do all these really impressive things.

13:01 The onus now to go and validate that this is true. And I have this challenge internally, which is people send me stuff and they do this deep investigation and they’ve used Argon to do it. And I’m like, my skepticism is actually a little bit higher today in this transitionary moment of people sending me stuff because I’m like this sounds compelling. I have no idea if this is actually true. And so, I’m actually curious how you all are, if you all have advice for folks as they use this model and now they start to do this really impressive stuff. How to do that if they don’t have 100,000

13:32 Google employees or collaborators to work with? How to do that introspection and get the confidence in the work that’s actually happening? - I mean, I actually find that this model will be good. It does not overclaim. It’s not too sycophantic. Like, it is as ground— I mean, we could do better in terms of being grounded, but it does not try to fabricate stuff, right? So I trust a lot of what it produces. Obviously, there’s misunderstandings, but there’s no misrepresentation of stuff.

14:04 - Well, first of all, welcome to my world. Just like the HBM memory story predating AI, I mean, I get pitched stuff all the time from like, very intelligent seeming, whether they’re folks who work with us or partners or companies wanting to do business. I mean, yeah, you need to have skepticism whether it’s an AI presenting you information or a person. It’s hard to tease apart. I mean, the cool thing is you can use the AI to check itself, whether it’s exactly the same model or a different model.

14:37 They generally don’t seem to have an ego about it in the sense that just because, if you use Argon to check Argon, it’s not going to be like, “Oh, that’s me.” They can actually check themselves. - Yeah, more independent. Yeah. - Internally, we have a whole family of models that we can use to also get diverse opinions nowadays. - Yeah. - Any other advice? - I was just going to say, we’re seeing in my neck of the woods, the bottlenecks are shifting.

15:07 So code reviews obviously is where a lot of focus is. People have talked about that. One thing I’m saying a lot to teams right now is I might get something I might suspect, like, this feels like— I’m not sure, this is— it just generated in one shot or something? So my go-to reply back is, share your prompt with me. Let’s refine it a little bit. Because I don’t want to discourage trusting the model, reaching for the model. But I also— you can’t just do your first one shot and be done with it. - Yeah. - So I think at least that’s the phase I’m finding

15:38 a little bit on the product side is people doing really interesting analysis or even ideas for product concepts. - I like this one actually, Josh. So you and I were talking in some other meeting earlier this week or last week. I don’t even know what day it is anymore. One of the threads of this conversation is this model really feels like there’s— and it’s why I’m excited for next iterations, like the harnesses, to see the model come to life in our products and other people’s products. But it feels like there’s a lot of— there’s iteration juice to squeeze. - Yeah. - I really am, as I’m building software, as I’m doing some work,

16:11 I continue to ask the model to do more and even very crappy prompts. Make this thing better. it actually goes and a lot of cases makes it better, which is really impressive. And so it’s interesting to see just out of the box without actually having to even have a really good perspective about what are the things that I want better. The model just goes and thinks and spends more tokens and creates something that’s really impressive. - But I think this is where the pressure gets higher to make sure that you are in fact doing the thinking and not outsourcing the thinking part. I think that’s what I find too,

16:42 that I’m having conversations with people, which is yes, absolutely. Use the model to aggregate the analysis, to give you perspectives, to riff with your perspectives, but don’t fully assume that everything there is canon. And I think what you find is that when you start asking questions of what you get back, you do start scrutinizing where the model might be off because you start looking at something and say, wait, that doesn’t actually sound right. So I think it’s definitely read the output yourself, make sure that the output makes sense to you,

17:14 question it, and then do another loop of iteration. Because I think oftentimes, actually, the places where the model fails, some are more nuanced and you might miss it. And especially if you’re not an expert, you’ll definitely miss it. But some are, if you scrutinize it, you’ll be like that seems a little off. And I think just asking that question gets you quite far. - Yeah, I love that. Sergey, we’ve had a lot of model iteration over the last few weeks, weeks and months, over the last six months maybe even. And obviously all of that has culminated with this moment.

17:45 So I’m curious, what— yeah, what were the combination of things that, made this Gemini 4 Argon moment possible? - Yeah, great question. And I would say that there are basically two things that came together. One was a deliberate decision to increase our cadence and to be able to use our own models to then make the next generation of models. The other is a generational leap that, you can’t quite schedule your breakthroughs.

18:17 If I could, I would have done it a little sooner. But, from Gemini 3 to 4, there’s a bunch of new technology in there. Gemini’s going to go from one version to the other, hopefully at an increasing cadence. - Yeah, that’s exciting. Maybe we also go around and talk about if anyone had any “aha” moments with Argon, like the first set of prompts or even maybe earlier predecessor, because we had tons of internal versions prior to Argon that you did the thing and you’re like, wow, this

18:47 is a model that I’m excited about and I want to get out the door. And I’ll tee off with my crappy easy example, which is I love making Minecraft clones. And so this is a fun— it’s because there’s so much nuance in the actual game. And so a lot of models would just give you something that like looks like Minecraft, but there isn’t depth. You go mine and look through the caves and there’s two levels deep, there’s just diamonds everywhere or whatever. And so there’s all these easy to— easy, very simple fast to grok,

19:18 is the model thinking deeply, which is why I like that example. And Argon crushed it for me. I did two prompts. The first one, visually everything was amazing. The second one added the depth to the game and the UI HUD that I really wanted. And so I was impressed. I was like, this is the model. I’m excited. Anyone else have any good ones? - Well, actually, before I talk about a prompt, which I’ll talk about in a second, one thing that I think was funny with this model, and this actually, I think, speaks to your point about just how we’re testing internally in cadence. Normally, I feel like when we have a new checkpoint,

19:49 we all bash it ourselves and start getting some vibes for it. This time we had this model and we almost immediately gave it to a set of Googlers to test. And I distinctly have this memory of, I think I was at home and a bunch of folks had started testing the model before I had even started testing the checkpoint. And so I’m looking at my phone and this chat group is blowing up. Of people being like, wait, this model is so good. I’m so excited about this. I just tried this use case. This is incredible. My life has changed.

20:20 And it’s that moment where you’re actually, it’s not even just you using the model and being excited about it. You’re like, something has shifted in the tenor. And that actually was a meaningful shift I think, from how we’ve heard. This is a critical audience of testers. This is not an audience that is just going to get excited because we put something out. And we’ve seen that criticism in previous earlier versions of the checkpoints. And so to see that shift. I think was actually a really interesting moment in and of itself. - I love that. - Yeah. - I think one thing for me more than a set of prompts is actually when

20:50 you’re running post-training, a lot of people come and pitch, “Hey, I need chips for this idea,” or “I need chips for this idea.” “I need 500, I need 1,000, I need 600.” Now, people come and I’m like, “I need Argon QPS.” And “I need ten QPS, I need fifteen QPS, I need like five QPS,” because they just want to use the model. They generate data with the model, they go and do their research with the model. So, it’s actually a shift in how we do post-training, because as Sergey said, the model actually is building the next version of the model. - That’s awesome.

21:21 - So I think that’s cool for us to be in this era. - Yeah, the one I’d add we had someone on the team who has a young kid, trying to teach him how to read. And so he had Argon go out, and basically his goal was to make a reading app to teach his son. But Argon didn’t just whip together some one-shot app with phonetic sounds. It actually went out, researched early childhood learning theory, incorporated that into an app with certain steps along the way, and then did the whole front end and got it, hosted it.

21:54 It was just this unbelievable end-to-end. Where it’s not just producing the site or the app at the end, but it’s all the stuff up front. And I think it speaks to a lot of the persistence and the goal-seeking that’s in the model now, which was a pretty interesting combination. It built its own knowledge base and then did the coding task, as opposed to just doing some coding task out of thin air. - I love that. And the eval is: how fast is this kid going to learn how to read? - That’s right. - So we’re checking on that. The TH sound. If they can’t read soon, we’re sending it back.

22:27 - I’ve seen it correct its own training data, which is impressive. It’ll go through and in training data, it’ll be asked to prove various things, mathematical theorems, and said, you know, this one’s wrong and here’s a counterexample. And just seeing it do that, it’s pretty impressive. - That’s awesome. One of the things that I’m excited about for this model is we were the first to launch one million input tokens. Maybe now we’re the first, I don’t know if this, if we’re the first for one million output tokens but industry-leading one million output tokens.

22:58 And I think it’s worth double-clicking on where we’ve actually seen, because we’ve done a bunch of tests internally. We’ve handed the model off to early access customers. They’ve given us feedback of where do you get the most value from one million input and output tokens, or one million output tokens in this case. So, maybe we can talk about use cases where this actually shines. - I think on some of the hard reasoning and math problems, we have a team or multiple teams going after open problems. And for those people, this long decode, the model really wants to go

23:29 long, and enabling the model to go to one million output tokens helps a lot. - I’ve asked Tulsee this question three times now, so I’ll ask it again in this forum as well, which is how– - We just do this. - How should somebody frame their thoughts of, as you describe that, I implicitly think of reasoning levels. And so how should I think about the difference between an extra high or a max reasoning level relative to this long, one million output token context length? - I think at high, which is our max, the model has the ability to really go to one million tokens if it can.

24:01 - Got it. - I don’t think it’s possible with competitor models, and we’ve seen a non-trivial percentage of trajectories hitting that limit on evals. - And I think that’s the key point, actually. Melvin keeps using the word enable, which I really like, which is the model is already capable of doing that, right? But if you only have the ability for 64K output, then the model has to stop, right? You have to sort of– it only has that capacity of what it can actually pull out. So, if you enable that to go longer, you’re now leveraging the model’s capacity to reason more,

24:32 especially at that high reasoning effort, to get you that full outcome. And that’s, I think, what we really want to enable. - Yeah. - One other comment on this is I will give the obligatory shout-out to all the Google infrastructure and research teams who are making this happen, because it is not easy to make the model do these very long horizon tasks and all the SDK API backend infra work that’s required to do this. It’s been impressive to see all the stuff that’s happening to enable this. - Also, shout out to our serving teams.

25:02 To ship a model like this to everyone we’re trying to ship it to, I think it’s quite the feat, and they’re hustling. - Yeah. - Maybe it’s worth a moment– Okay, just to clarify, I think when we say now a million output tokens, we mean over a single step because our old 3.8 Flash could do a million tokens if it does this step, then that step then the other step. - Yes, correct. - So we’re talking about in one step.

25:33 And I think that’s for things it would have to think very deeply about. So typically not like a coding problem because in a coding problem, you’re looking at this piece of code, writing a bit, running it, whatever. But for a very hard math problem, for example, that’s where you might need to think for quite a long time. And if you think about it in human terms, maybe you can think at a few tokens a second. I don’t know. But a million tokens,

26:04 if you think about that as half a million seconds, that’s like a full week or so, plus or minus. Not to mention if you sleep, and eat and do other things, then maybe it’s we’re talking about like a month of thinking. It’s an enormous amount of capacity. - I like this analogy. A month of thinking in a single request. - I like it. - That’s powerful. - Yeah, it’s a good pitch. - Something that’s actually worth talking about, and Josh, maybe actually some of the writing examples underscore this, I think for coding as well

26:35 that we’ve seen so far, but people actually like working with the Argon model. And so this alignment behavioral aspect that folks have been driving, I’m curious if anyone wants to talk through the high level, what we’ve done, why it’s great, what the internal feedback has been on this. - So, a few things, right? When we think about behavior, we actually think about it across a number of different axes of how the model is engaging with you. And I think it’s important because you can get to either extreme and neither is particularly good, right?

27:08 So if you talk about for example, we talk a lot about how this model is persistent. And I think that’s a really positive trait in that the model works really hard and it keeps going. - Yeah. - You can take that to an extreme where the model is, too persistent and goes too far without checking in with you or, asking questions around ambiguity or clarifying. You could also go to the other extreme where the model is annoying because it is just, like asking you questions again and again and again, right? And so, like, I think we think a lot about how do you find the right middle ground of that type of trade-off, right? And so we’ve push the model to be persistent.

27:40 And so we definitely are anchoring more to that angle, but we want it to be able to clarify ambiguity. Similarly, another area that we’ve invested in for this model has been transparency. You also want to again, find the right balance between being too verbose and not telling the user anything at all. And so one of the things we’ve tried to invest in is the model communicating more about what it is doing. And so these are changes that we’ve thought through intentionally both in terms of the evals and how we’ve been actually measuring the model, but also in training. And we’re seeing that in Googler feedback,

28:12 which is also really nice. So we’ve been actually, using Argon to compile feedback across, dozens and dozens and dozens of chat rooms, trying to pull out and parse feedback. And then we actually run live experiments. And in those live experiments, we actually evaluate for Googler implicit signals on how they’re engaging with the model. And that also helps us understand where is the model balancing, in these various places? - And the Googlers have opinions. - The Googlers have opinions. - They do, in a good way. I mean, I think this is where it intersects the product in interesting ways.

28:42 Because even if you’re in Antigravity or the Gemini app, we’ve done tons of experiments on how long should those thought traces be and how many should show up. And when do people hit stop or complain at the model? So you look at all these different things and think through what’s the right UI. Trade-off when it comes to that transparency too. Interesting, what we tend to see is early on in a new model, people really value deep, frequent thought traces. And then over time, they either grow to trust the model more or they don’t, and they start to look at that less.

29:14 And so we’ve played around a lot with how much to show over the lifecycle of the user’s engagement with the model. - I love that. That’s awesome. Yeah, well, I want my personalized Gemini app experience with thinking levels like that. I think we’re actually shipping a new thought summarizer in all the— at least in the API is shipping. I don’t know if this is in— if it’s already in Gemini app or not, but it’ll be interesting to see what people think of the more concise thoughts. More concise thoughts. Should be interesting. - Better quality, I think. - This is awesome. Thank you, all four of you, for sitting down.

29:46 I feel like Gemini 4 Argon feels like a huge step, push the ecosystem forward, and just the start. More new models to come the rapid iteration. Flywheel is in full effect, which is exciting. And I think hopefully we’ll see lots of great new feedback that we haven’t seen yet and we’ll be able to take action on that as well. So, thank you all - yeah, please share your feedback. - Yeah, please send us feedback. - We’re trying to model. - Awesome. And thanks everyone for tuning in to this episode of Release Notes. We’ll see you in the next one.