How VAST and NVIDIA Are Delivering Confidential AI

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: How VAST and NVIDIA Are Delivering Confidential AI · Channel: VAST Data · 04:23 · Watch on YouTube

Topics: AI agents, Data, databases and embeddings, AI infrastructure and hardware, Privacy and data protection, AI safety

Summary

VAST Data and NVIDIA discuss their partnership to build an AI data platform that supports secure, governed AI at scale. They describe how enterprises need to connect AI models to sensitive data without moving that data out of protected environments, while model providers also need to protect their proprietary weights. The conversation presents confidential computing—running encrypted model weights in encrypted virtual machines, with software attestation—as a way to address both concerns. The speakers envision collaboration across model companies, server providers, AI clouds, and enterprises, enabling secure deployment and fine-tuning of models at the edge and in data centers.

Key points

VAST and NVIDIA have worked together to develop an AI data platform, which they describe as an AI operating system.
The platform is being adapted for CUDA- and GPU-accelerated processing of structured and unstructured data.
Enterprises need a governed and secure way to connect AI to their internal data.
Some regulated organizations cannot move sensitive data, such as healthcare records or financial transactions, to the cloud.
Confidential computing can bring encrypted model weights into a data center and keep them encrypted until they run in an encrypted VM.
GPU support for encrypted memory and software attestation are described as foundations for protecting model weights.
The speakers say model companies, server companies, AI clouds, and platform providers must collaborate to expand AI deployments.
Enterprises could fine-tune models and run them securely, protecting the intellectual property in those models.
The discussion anticipates companies using large numbers of agents that learn over time.

Chapters

Start	Chapter	Summary
00:04	The VAST–NVIDIA partnership	The speakers introduce their partnership and discuss evolving the AI data platform to support secure, GPU-accelerated AI.
01:08	Connecting AI to enterprise data	They explain why enterprises need secure, governed access to their data and why some cannot move it to the cloud.
02:10	Confidential computing and model protection	The speakers describe encrypted model weights, encrypted VMs, GPU capabilities, and software attestation as components of confidential computing.
03:16	Collaboration and enterprise AI	They discuss collaboration among AI providers and the prospect of enterprises securely deploying and fine-tuning models, including models used by agents.

Quotes

AI has finally become useful. — 01:08

they don’t want to give up their data. — 01:40

the model provider can be assured that their model weights can’t be seen by the operators of the data centers. — 02:44

Everybody needs to collaborate in order to make this work — 03:16

that becomes their competitive advantage. — 04:23

Concepts and entities

VAST Data, NVIDIA, confidential computing, encrypted model weights, encrypted VM, AI factories, Cohere, Fundamental, Neumotoron

Full transcript (original language)

00:04 Hey Ren, good to see you. >> Hi, good to see you, too. Thank you for having us. >> Thank you for the very, very long partnership that we’ve now had. >> The partnership between NVIDIA and VAST has been amazing. I think it was 3 years ago we decided to reinvent the data platform. >> Yeah, we built the data platform, what we call the AI operating system, and over time that evolves into managing models and making them safe and secure. And so, all of these aspects of >> [music] >> data security, weight security, making sure that everybody can work together is becoming a big part of what’s

00:35 unlocking the value of AI. >> I know there’s been a trillion dollars projected to be building out of AI infrastructure over the next uh few years. Uh and uh data platforms are foundational uh to everybody building AI and running AI. And so, the fact that you guys have reinvented your platform to be CUDA-accelerated, GPU-accelerated, both in structured and unstructured processing, uh really makes sure these AI factories can be fully utilized globally. >> And as [music] these AI factories grow and scale and require more compute and more networking, uh there’s no better

01:08 partner than NVIDIA. Let’s talk a little bit about where this is going. Uh the enterprises are a little bit struggling to adopt AI. What Why do you think that is? >> Well, in the last 6 months it’s been pretty phenomenal. Uh AI has finally become useful. But for AI to truly unlock the value of enterprises, you’ve got to connect it into enterprise data uh in enterprise uh data sources. Um but for many enterprises, they just don’t know how to do that in a governed and secure way. Uh there’s been kind of this false trade-off of do I use cloud APIs? What’s the security model around me

01:40 moving my data to the cloud? Um and so, I think that’s the next big evolution. How do we securely connect the world’s AI into the knowledge that’s stored within enterprises? >> Yeah, the enterprises, they don’t want to give up their data. In some cases, they’re not allowed to. But the model builders also don’t want to give up their weights. >> Yeah, well, regulated markets, whether we’re talking healthcare, finance, uh government agencies, um they can’t move that data from their environment to the cloud. Uh and it’s because uh you know, whatever whether it’s financial

02:10 transactions, healthcare records, or government contracts, >> [music] >> that the information’s too proprietary and too confidential. And so, the only way to uh unlock AI with that data set is really to bring the model down to the data center. Um and that’s where I’ll say 60 to 70% of all the world’s data stored within that enterprise. So, every model builder now is thinking about like how do I bring my model into the data center? And the best way to do it is through confidential computing. Confidential computing allows us to bring >> [music] >> um encrypted model weights into the data center, and those encrypted model weights are never decrypted until

02:44 they’re running in an encrypted VM. In this way, the model provider can be assured that their model weights can’t be seen by the operators of the data centers. >> And that requires both software and hardware underneath to make it possible, right? >> That’s right. I mean, we’re as you know, an accelerated computing company. We build the foundations for things like confidential computing. We build the capabilities to have uh GPUs uh run these models in encrypted memory, and then we ensure that the software can be attested and verified before the model weights are decrypted, but that requires us to work with platform companies like

03:16 yourselves that then provide ultimately the governance layer for running these models at scale and at the edge. >> So, it sounds like we need the model companies, we need server companies, we need AI clouds. Everybody needs to collaborate in order to make this work and increase the size of the AI deployments. >> That’s right. And you guys I I think uh are that ideal platform, right? To be able to bring these confidential enclaves for all the model vendors uh to bring their models securely to the edge. >> Uh we’re working [music] with companies like yourself, with Neumotoron building your models, and Cohere, and Fundamental, and all

03:50 working together to increase uh this [music] the speed at which we can go. >> And what’s awesome is any model company that spends any amount of money in building [music] this proprietary IP can deploy it to the edge. It also gives enterprises the ability to post train those models and then run them securely in any data center around the world. >> Yeah, I think today it’s the model companies that own the weights, but tomorrow we want every enterprise [music] to fine tune their own models and do reinforce learning and run agents and over time that IP that’s in those models is secure for them as well.

04:23 >> Yeah, that becomes their competitive advantage. >> I think it becomes everything. If today we have a company that has a thousand people, tomorrow they’ll have a million agents and >> [music] >> over time they learn things and that’s what that company is. Thank you again, Justin, for having [music] us here. >> And thank you, Renen, for the partnership between Nvidia and Vast. It’s been amazing.