Why "best model" is the wrong question now

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: Why “best model” is the wrong question now · Channel: LangChain · 00:34 · Watch on YouTube

Topics: AI agents, Software engineering with AI, Model inference and optimization, AI models

Summary

The speaker describes a new generation of models making large, continuous gains in capability. As more software engineering tasks become easy for all models, developers are shifting from seeking the most capable model to managing the cost of coding agents. Some workloads still require frontier intelligence, particularly competitive or adversarial ones. For many software tasks, once models are good enough, speed and serving cost matter more.

Key points

The speaker characterizes Fable 5 as part of a new generation of models with continuous step changes.
Capability gains are increasingly making software engineering tasks intelligence-saturated.
Many models can already perform some software tasks well.
Developers are increasingly concerned about the cost of coding agents.
Some workloads still call for the most frontier intelligence, especially competitive or adversarial ones.
For many software workloads, performance past a sufficient threshold matters less than speed and cost.
The speaker reports strong developer and company demand to optimize serving costs.

Chapters

Start	Chapter	Summary
00:00	Continuous gains in model capability	The speaker introduces a new generation of models and argues that their capability gains are changing the software engineering landscape.
00:34	When frontier intelligence matters	Some workloads require the highest intelligence, particularly when competing against an intelligent counterparty.
00:34	Optimizing for speed and cost	For many software tasks, once models are good enough, developers focus on speed and serving costs.

Quotes

it’s actually no longer just about the capability increases. — 00:00

literally all of the models can do them well. — 00:00

I’m spending so much money on my coding agents. — 00:00

what you care about is speed and cost. — 00:34

How can I now optimize this serving cost? — 00:34

Concepts and entities

Fable 5, coding agents, serving cost, frontier intelligence

Full transcript (original language)

00:00 I think now we see, you know, with like Fable 5, it’s like a new generation of models, continuous step changes. And now the step changes are so high, it’s actually no longer just about the capability increases. I think we’re kind of seeing the opposite trend in some sense, which is more and more of the tasks in software engineering are getting intelligence saturated. So like literally all of the models can do them well. The sensitivity of a lot of developers has totally shifted from, wait, I need to be using the best possible model to holy cow, I’m spending so much money on my coding agents. How can I be a little bit more efficient, you know, about this, I think this is like an underappreciated aspect of continuous gains in frontier intelligence,

00:34 which is that, you know, for some workloads, you always want the most frontier intelligence, especially if it’s like an adversarial game between parties and your intelligence has to be higher than the counterparty’s intelligence, like if you’re trading in competition or something. But for a lot of software we want to build, the saturation threshold where once it’s good enough, what you care about is speed and cost. And I think that’s actually where we see a huge amount of demand from developers and the companies we work with is, okay, this already works. How can I now optimize this serving cost?