Gemini 4 Argon (In-depth benchmark analysis): Did Google make a big comeback?

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: Gemini 4 Argon (In-depth benchmark analysis): Did Google make a big comeback? · Channel: AICodeKing · 14:08 · Watch on YouTube

Topics: Task and workflow automation, AI evaluation and benchmarks, Software engineering with AI, Markets and economy, AI models

Summary

The video examines Gemini 4 Argon using published results from independent benchmarks and Google’s own evaluations, while noting that the presenter has not tested it firsthand. Argon shows strong results in business workflows, spreadsheets, long-context tasks, and willingness to admit uncertainty, but its coding performance varies across benchmarks. The presenter also highlights limits in the comparisons, including different evaluation methods and inputs. Argon looks worth trying at its introductory price, but its standard pricing and uneven coding results make the presenter cautious about adopting it as a primary tool.

Key points

Argon’s initial rollout is limited, and Google has not announced a public release date.
It scores competitively on the Artificial Analysis Intelligence Index and leads the Valse Index, though Valse weights finance heavily.
Argon outperforms Sonnet 5.5 on Zapier’s business-automation benchmark, but only about half of the tasks pass every check.
Its spreadsheet benchmark result is strong, but the reported score is rubric-based rather than a percentage of perfect completions.
Coding results are mixed: Argon leads on one software-engineering benchmark but trails competitors on others.
Artificial Analysis reports a low hallucination rate among high-scoring models, alongside lower knowledge accuracy than Astra.
The million-token output limit and long-context results could help with demanding tasks, but their practical value depends on output quality and cost.
API prices rise from $2 input and $10 output per million tokens to $4 and $20 after the introductory period.
The presenter recommends testing Argon on specific difficult tasks and comparing total costs rather than rebuilding a workflow around launch charts.

Chapters

Start	Chapter	Summary
00:02	Launch, past concerns, and access	The presenter introduces Argon’s headline results, explains caution based on earlier Gemini releases, and describes the limited initial rollout.
02:47	Independent results for professional work	The video reviews Argon’s reasoning, business-workflow, and spreadsheet results, while explaining what those benchmarks do and do not establish.
05:28	Mixed coding results and hallucination behavior	The presenter compares coding evaluations, discusses a legal benchmark discrepancy, and explains Argon’s reported tendency to admit uncertainty.
08:42	Long outputs, context, and specialized benchmarks	The video considers Argon’s output limit, long-context and video results, and cybersecurity evaluations with differing methods and costs.
10:52	Pricing and overall verdict	The presenter weighs promotional and standard API costs, recommends practical task-by-task testing, and concludes that Argon is a credible but uneven contender.

Quotes

This is an analysis of their published testing. I haven’t run my own hands-on tests with Argon yet. — 00:37

The model saying that it completed the task doesn’t earn it a pass. — 04:23

So, 15% doesn’t mean 15% of everything Argon says is false. — 07:36

That changes the comparison quite a bit. — 11:25

At the introductory price, Argon looks worth trying. — 13:33

Concepts and entities

Gemini 4 Argon, Google, Artificial Analysis, Valves Index, Zapier, Surge, Arena, CWE Bench, Google AI Ultra

Full transcript (original language)

00:02 [music] >> Hi. Welcome to another video. So, Google has announced Gemini 4 Argon, and there are some pretty interesting results here. It leads an independent benchmark for business workflows, takes the top spot on the Valves index, and looks much better at admitting when it doesn’t know something. But, the coding results are mixed, and the pricing has a catch that changes the value comparison quite a bit. I’ve gone through Google’s announcement, evaluation footnotes, and results from artificial analysis, Valves, Zapier, Surge, Arena, and the people behind CWE

00:37 bench. This is an analysis of their published testing. I haven’t run my own hands-on tests with Argon yet. My initial take is that this looks like a serious option for professional work with enough weaknesses that I wouldn’t move my entire coding workflow over just because of the launch charts. So, let’s get right into it. And there’s a reason I’m cautious with Google. I’ve been disappointed by some of the previous Gemini releases, especially when the impressive reasoning scores didn’t match what I wanted from a coding agent. With Gemini 3.1 Pro, my one-shot KingBench result was actually very good

01:10 at 96% but my agent testing had repeated planning problems using the tools, and basic coding mistakes. On one go task, it duplicated a method and caused a compiler error. That’s the kind of thing a headline reasoning score doesn’t tell you. Those were particular runs in a particular setup, but they were enough to make me question its usefulness for my workflow. Then Gemini 3.5 Flash disappointed me on value. Its standard prices were one par at $50 for input and nine tars for output per million tokens, compared with 50 cents and $3 for Gemini 3 Flash.

01:45 That’s three times the token price. For something carrying the Flash name, I wanted a much stronger reason to pay that difference. And there was the wait for Gemini 3.5 Pro at I/O in May. Google said it expected to roll it out the following month. By its July 22 earnings call, it was still in testing. Now we’re looking at another flagship announcement with limited initial access. I understand taking more time to get a model ready, but the earlier timeline gives me a reason to wait for actual availability before getting too excited.

02:15 So, Argon has some confidence to earn back. That makes the independent results especially useful here. I want to give Google credit where it has improved while checking whether it has fixed the things that made those earlier releases frustrating. First, access. The initial rollout is through Google’s Fairwind program for selected cybersecurity partners. Google says wider access will start with paid API customers and Google AI Ultra subscribers, but the announcement doesn’t give a public release date. So, buying a subscription today shouldn’t be treated as a guarantee that you can immediately use Argon. Now, let’s start

02:47 with the independent results because these help put the announcement in context. On Artificial Analysis, Argon had high reasoning scores, 53 on the Intelligence Index. That’s the same rounded score as GPT-6 Astra at max reasoning and one point above GPT-6.1 Sonnet at max. Opus 5.5 scores 58 and Sonnet 5.5 scores 56, both at max with default fallback routing for refusals. So, Google is competitive here, although Anthropic is still ahead on this particular overall measure. Also, these are index points.

03:20 A score of 53 doesn’t mean it completes 53% of your work. And a one-point lead doesn’t mean you’re going to notice an improvement on every task. The Valse Index gives Argon a stronger result. It scores 68.9% ahead of Sonnet 5.5 at about 67% and Opus 5.5 at about 67%. Valse combines finance, coding, legal, and tax benchmarks with weights based on those sectors’ shares of the US economy. One detail I like here is that Valse already uses Argon’s higher standard token prices in its cost calculation.

03:51 Even then, it reports about $15.68 per test versus 21.34 for Sonnet and $32.14 for Opus. So, there’s a real value argument within this suite. But, finance accounts for a little over half of that index’s weighting. If most of your day is fixing code, that overall ranking might not reflect your priorities. I’d look at the coding components before deciding which model to use. Zapier’s automation bench is another strong result. Argon at high reasoning gets 51.29% compared with 44.75%

04:23 for Sonnet 5.5 at max with its default fallback routing. This benchmark checks whether an agent actually completes workflows across simulated business apps. That means checking the records and other changes it leaves behind. The model saying that it completed the task doesn’t earn it a pass. Every required check has to succeed, which I think is a useful way to evaluate automation. Argon is ahead here, and that’s good to see. But, roughly half the tasks still don’t pass every check. For me, that supports using it to assist with these workflows while checking the results. It doesn’t establish that you can leave it to run a business without supervision. There’s

04:55 another independent win on Surge’s GDP.xlsx spreadsheet benchmark. Argon scores 38.3% compared with 30.3% for Opus 5.5 and 29.1% for Sonnet 5.5. This is a rubric-based score, so don’t read it as the percentage of spreadsheets it completed perfectly. Taken together, those results make the professional work pitch look pretty credible. I’d be interested in testing it on a messy workbook where it has to follow references across sheets and explain how it arrived at the answer. That’s the sort of task where checking a few easy cells isn’t enough. Anyway, now let’s

05:28 come to coding because this is where the picture gets more complicated. Google reports 77.9% on Deep Way U version 1.1 ahead of Opus 5.5 at 74.2% and Astra at 74.1% in its comparison. That’s a strong result. But, on Frontier’s way version 2, Argon gets 55% while Astra gets 65.5% and Opus gets 62.3% and on Google’s Terminal Bench 4 comparison, Argon is at 57.4%

06:00 versus 66.4% for Opus. So, even within Google’s own table, the answer changes depending on which software engineering test you look at. There’s a methodology detail here, too. Google computed Argon’s Deep Stew AU result with Mini Stew AU agent, while Astra’s comes from the benchmark leaderboard and Claude’s from system cards. These are separately published evaluations. I’d want a controlled comparison in the same agent setup before attributing every difference to the model. Vala independently puts Argon second on both its app building and code migration benchmarks

06:31 behind Sonnet 5.5. That is encouraging, but the app building gap is less than half a percentage point, smaller than the reported standard errors. I wouldn’t make a big deal out of who is first or second there. For coding, my reading is that Argon deserves testing on substantial tasks, especially migrations and app building. The results don’t give me a reason to assume it will consistently beat your current agent on debugging, terminal work, and difficult repository changes. This is also why I wouldn’t just count how many rows Google wins. A chart reading improvement and a repository fixing improvement aren’t interchangeable when you’re choosing something for your daily work. One more

07:03 example is the legal result. Argon’s 19.6% looks dominant against the models Google selected for its chart, but Vala’s own Argon report lists it fifth for the Harvey legal evaluation. That doesn’t erase the result. The launch comparison simply doesn’t establish that it leads the whole field. Now, the hallucination result is probably the most interesting part of this release for me. Artificial Analysis reports a 15% hallucination rate on AA Omniscients, the lowest among the models scoring at least 45 on its intelligence index. Astra at max is at 51%. However, Argon’s

07:36 knowledge accuracy is 50% compared with Astra’s 63%. Those numbers can sound contradictory until you check the definition. The hallucination metric looks at the responses that weren’t correct and measures how often the model gave an incorrect answer instead of a partial answer or an admission that it didn’t know. Correct answers aren’t in that denominator. So, 15% doesn’t mean 15% of everything Argon says is false. It also doesn’t mean it has solved hallucinations. The useful result is that it is more willing to acknowledge gaps even though it answers fewer of these questions correctly than Astra. Personally, I like that trade-off for

08:09 research. If a model can’t support a claim, I’d like it to say so. A confident answer with a made-up API or reference can waste a lot of time, but I still want to check how that behavior carries over to browsing and coding where the model has tools available to verify things. Arena gives us a different kind of evidence. Its September 30th text leaderboard puts Argon high at number one with a score of 1525 and around 4,900 votes. The result is marked preliminary. That’s encouraging for how people respond to its answers. It also helps explain why a model can look excellent in chat without

08:42 topping every technical benchmark. Human preference, factual knowledge, and successfully changing a code base are separate things. I’d use the Arena result as another signal alongside the task-based tests. Now, Google also highlights a million token output limit. The output part matters. This gives a reasoning trajectory much more room to continue working. It isn’t just a claim about how much material you can put into the prompt. Artificial Analysis says it tested a feature called long decode continuation, which pauses long responses and resumes them through follow-up calls, allowing reasoning to reach that limit without request timeouts.

09:15 So, the practical question is whether that extra thinking produces a better result at a cost you’re happy with. I don’t need a model to write a million tokens for a small bug fix, for a difficult optimization problem. I’d be more willing to spend that time and money if it produces something I can verify. There’s a separate long context result, too. Google’s reported graph walks result covers context between 256,000 and 1 million tokens. Argon scores 84.2 on the F1 metric compared with Astra’s 71.8. That’s promising for following relationships across long inputs,

09:48 although one graph benchmark can’t establish how well it understands every large repository. And the video results need a footnote. Google reports 91.7% on LVBench, but its methodology uses one frame per second for Gemini and fixed frame caps for competitors because of their API limits. That can be a practical advantage for the Gemini setup, but it isn’t an equal input test of visual reasoning. Cybersecurity has some similarly useful detail. On CWE Bench version 1, Argon ties Astra and Grok 4.7 at 68% under the programmatic

10:21 single attempt measure. Under the separate judge panel measure, Argon gets 62% while Opus gets 67%. The benchmark also lists $63 per rollout for Argon and 79 cents for Opus. Those figures use different agent harnesses, and the Argon calculation uses a cached input price that differs from Google’s introductory discount. So, I wouldn’t turn that into a universal cost ratio. But it does show why the score, grading method, and spending all deserve a look before calling something the best

10:52 security agent. Now, let’s talk about the API pricing because this could change whether Argon makes sense for you. Google advertises $2 per million dollar input tokens and $10 per million output tokens with a 95% discount on cached input. The footnote says the prices become $4 and $20 after the introductory period. Google hasn’t given an exact end date in the announcement. That is a doubling of the input and output rates. If you’re budgeting for an application, I would calculate the cost at the standard rate and treat the launch price

11:25 as a temporary benefit. Artificial analysis measures about 199 per intelligence index task at the promotional rate. It calculates $3.98 after the discount ends. Astra at max costs about $3.26 on that same evaluation, while GPT-6.1 Soul at max costs about $0.72. That changes the comparison quite a bit. Argon looks cheaper than Astra at launch, but more expensive after the promotion on this workload. Soul is much cheaper while scoring only one index point lower. Your own task mix could produce a different result, but

11:59 you shouldn’t choose based only on the price per million tokens. Artificial Analysis also reports around 62,000 output tokens per task for Argon, compared with around 27,000 for Astra. So, that lower unit price is doing a lot of work. The amount the model generates matters, too, and efficiency isn’t the same across evaluations. Remember that Valves still puts Argon below Sonnet and Opus in cost per test using its standard price. That’s why I want to measure the bill for the actual job, how much it costs to get a patch accepted or a work book

12:30 checked, including any retries. I also wouldn’t promise that Argon feels fast in an editor yet. Artificial Analysis doesn’t publish an output speed measurement for it on the model page. Valves has task duration numbers, but those don’t tell us how responsive the eventual public service will feel while you’re working. So, how good is Gemini 4 Argon looking overall? I think the strongest case is professional work where it needs to follow information across documents and tools, plus tasks where admitting uncertainty is useful. There are several independent results

13:01 supporting that case, so I wouldn’t dismiss this as a model that only looks good in Google’s own charts. For coding, I’m more cautious. It has strong results in some areas and clear losses in others. I try it on a migration or a difficult task that my usual model struggled with, then compare the finished result and the total cost. I wouldn’t assume a new model name means I should rebuild my workflow around it. The things I want to test when access opens are fairly practical. Can it finish changes without quietly dropping requirements? Can it recover after a failed approach? And does giving it more

13:33 time actually improve the patch enough to justify the extra tokens? At the introductory price, Argon looks worth trying. At the standard price, I would be more selective about which jobs I send to it. The evidence gives Google a credible contender here after the earlier disappointments. Consistent results in the tools we actually use would do the most to win back my confidence. Overall, it’s pretty cool. Anyway, let me know your thoughts in the comments. If you like [music] this video, consider donating through the Super Thanks option or becoming a member by clicking the join button. Also, give this video [music] a thumbs up

14:08 and subscribe to my channel. I’ll see you in the next [music] one. Until then, bye. >> [music and bell]