AI Debate Needs Evidence, Not Hopes and Fears: deSouza

Author: Bloomberg Television · Watch on YouTube

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: AI Debate Needs Evidence, Not Hopes and Fears: deSouza · Channel: Bloomberg Television · 06:50 · Watch on YouTube

Topics: AI evaluation and benchmarks, AI governance and regulation, Local and open-source models, AI safety

Summary

The interview argues that AI models need comprehensive, independent evaluations to improve public confidence and help governments and companies understand their capabilities and risks. Francis says decisions about regulation and development speed should be based on evidence, not hopes or fears, and that mitigation requires first knowing what risks models pose. He describes competing views on whether AI’s potential benefits justify rapid development or whether its risks require slowing down, and says the right approach is to move responsibly based on assessments. He also emphasizes that evaluation must cover open-weight models as well as frontier models, and examine more than accuracy, including bias and whether systems follow appropriate steps.

Key points

The White House agreement included commitments to increase companies’ evaluations and have independent evaluators test models.
Independent assessments can help rebuild public confidence and give governments a clearer view of models’ capabilities and risks.
The interviewee’s business provides data for model development, helps deploy agentic workflows with guardrails, and evaluates models for governments and enterprises.
He describes the company’s role as providing a reliability layer for AI.
AI development and regulation should be guided by evidence about model capabilities and risks rather than hopes or fears.
He calls for comprehensive testing across domains including cyber, biology, and radiology before deciding on appropriate mitigations.
The debate includes those who prioritize moving quickly to realize AI’s benefits and those who favor slowing down until risks can be mitigated.
A comprehensive evaluation strategy should include open-weight models, not just frontier models.
In clinical settings, evaluations should consider whether a model followed the steps to reach an answer, not only whether its diagnosis was accurate.

Chapters

Start	Chapter	Summary
00:00	The case for independent model evaluations	The interviewee supports more visibility into AI models, including company evaluations and independent testing of models across the field.
02:04	An evidence-based approach to AI risks	He argues that the pace of development and regulatory responses should follow evidence about models’ capabilities and risks.
03:40	Balancing AI’s benefits and risks	He describes opposing views on moving quickly or slowly and says responsible decisions depend on understanding and mitigating actual risks.
05:17	Testing open-weight models and clinical systems	He calls for evaluations of all models and says assessments should examine risks, bias, and whether systems follow appropriate reasoning steps.

Quotes

we need sort of an evidence-based strategy here — 02:36

We need the visibility first is what we’re pitching for. — 03:08

we can’t mitigate something if we don’t fully know what the risk is. — 04:44

it’s not just about testing the frontier models, it’s about testing all the models — 05:48

Concepts and entities

independent evaluations, frontier models, open-weight models, Agentic workflows, Dario Amodei, Jensen Huang, Wall Street Journal

Full transcript (original language)

00:00 We saw this gathering at the White House of President Trump and the big AI and tech executive signing on to this voluntarily agreement to do some self-policing, internal checks, external assessments of those checks. Is it adequate? Is it enough? >> So first of all, thank you for having me. I think what we are we’re all seeing is it’s becoming clear that we do need additional visibility into what these models can do and that we need additional security and guardrails as these models are developed and they are released. And so what we’re seeing are

00:31 steps in that direction. I think coming out of the accord, some of the things that that the companies committed to yesterday was to increase their own evaluations of their models, but then also to have independent evaluators test the models. I think it’s important in terms of rebuilding confidence from the public, but as well as providing safety for governments to have independent views into what the models are capable of and what the risks associated with each model are. And this isn’t just for the frontier models. This is the open

01:02 weights models. This is all the models that exist. >> Mhm. Francis, I’m curious, what’s better for your business? And maybe you just answered that. If there’s no regulation coming out of Washington, does that mean more growth cuz companies need your testing before models get out the door or the government will need your testing because there’s no regulation to to control the release of these models? >> So what we do as a business is one, we provide data to the model companies to help develop their models.

01:32 So as we continue to see more models develop, the demand for data continues to go up and as the models get more sophisticated, there’s the more demand for data. We also work with governments and companies to deploy agentic workflows and put guardrails around those workflows to enable them to do more more cheaply and work across models and across clouds. So we do that and then the third is we provide evaluations for both governments and enterprises to evaluate, you know, what the models can do. So, you know, from our perspective, you know, we are about providing the

02:04 reliability layer for AI, and whether the governments do it or companies do it, you know, that’s that’s our business, and we expect that business to be continue to be more important for companies as you deploy more AI in your organizations. >> Well, so for all of the data you’ve collected, all of the evaluations you’ve conducted, do you see merit in the arguments that some of these companies themselves are making about the need to pace the technology? Is it developing and advancing to such an extent that it does pose a grave threat to the public? >> From from our perspective, you know,

02:36 what we believe is that we need sort of an evidence-based strategy here, which means instead of basing our our strategy, our regulatory strategy, or our even the pace at which we develop models on on hopes and fears, we really need to get a clear understanding of what the models are capable of, what the risks are, and then use that to guide what mitigations we take. So, we need to understand is there a part of the models that are getting ahead of our ability to mitigate against them, and then take appropriate actions. >> And is there Are we there yet?

03:08 >> We need the visibility first is what we’re pitching for. We say, “Look, we need comprehensive testing of these models to understand their capabilities and then their risks across multiple domains, cyber, biology, radiology, and and that’s what we need first to understand what are the actual risks that these models bring, and then act on it.” But without visibility, we’re sort of, you know, moving around in the dark trying to understand what it is we’re actually trying to mitigate against. >> Francis, we’re seeing some remarkable headlines across the terminal on behalf of the Wall Street Journal

03:40 here from the Wall Street Journal about that gathering Kaylee mentioned today at the White House, which was a who’s who of AI including Dario Amodei, the first one to write the essay that got the whole debate reinvigorated most recently. Amodei was confronted the journal reports by other AI executives by his comments at the White House questioning exactly what it was he said. Even Jensen Huang questioning why he was so extreme in public about the potential dangers of AI.

04:12 Did Dario Amodei go too far based on what you’ve seen? >> Look, here’s what we’re seeing. We’re seeing very smart people have very different opinions on this topic. And And they’re both valid to be honest. One group is saying the benefits that AI can bring are so important in terms of advancing our understanding of disease, you know, helping combat climate change that we have a moral obligation to move as quickly as possible. 10 million people a year die of cancer. If we can help develop, you know, cures and understand the disease better, we need to move as quickly as possible,

04:44 responsibly. The other side is saying, “Look, it’s very risky and so we need to move more slowly until we can mitigate the risks.” And the reality is, you know, the truth is somewhere in between. We need to actually understand the risks that are posed. And until we do those assessments and say, “This is the actual risk proposed that that that the models can can and can bring.” And And then we mitigate them. But we can’t mitigate something if we don’t fully know what the risk is. Trying to slow things down without fully understanding why is also not the right thing to do as is going too quickly if

05:17 we know that there’s a risk. So, again, our approach is let’s have an evidence-based discussion, which is let’s understand what the models can do, what the risk is, and therefore what the right the right mitigation is because we all want to move as quickly as we can, as responsibly as we can. >> How much of your assessment has focused on open weight models like what China is uh focusing on? And do you see them posing greater risks like we are talking about? I just wonder what you what your comparison is of the two different

05:48 systems. >> Yeah, I I actually think that’s a really important point and I think open weights models haven’t been enough in the discussion. A lot of the conversation today has been around, you know, assessing frontier models or pacing the frontier. But I think we’re missing a the big point, which is a lot of people are using open weights models and we need to understand the capabilities of those models and the risk that those models pose too. So, a comprehensive strategy here has to cover all models to say, it’s not just about testing the frontier models, it’s about testing all

06:19 the models and understanding the risks that are posing. The risks could be across a number of fronts. It could be cyber risk, it could be but it could also be the biases that are built into the model and and how their decisions are made. You know, we’ve published some evaluations that show that in a clinical environment, for example, just evaluating models on the accuracy of a diagnosis isn’t enough because some models can get there by skipping a bunch of steps and guessing the answer. And so, when you’re evaluating, you know, a clinical model, for example, you look at accuracy but you look at a whole bunch

06:50 of other facts including did they follow all the steps to get to the right answer cuz if they didn’t, that’s not good enough. And so, whether it’s a frontier model or an open weights model, you need to run those evaluations and I don’t think that’s part of the conversation enough.