3. What is Databricks Unity AI Gateway and how it works
Read the Portuguese translation →
Author: Databricks · Watch on YouTube
Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.
Original language: English
Original video: 3. What is Databricks Unity AI Gateway and how it works · Channel: Databricks · 01:36 · Watch on YouTube
Topics: AI evaluation and benchmarks, Model inference and optimization, Markets and economy, AI models
Summary
Section titled “Summary”Unity AI Gateway provides a central control plane for managing access and spending across models, agents, MCP servers, and tools. By placing multiple models behind one endpoint, teams can change or test models without changing the application or affecting users. Requests are logged in system tables, allowing teams to compare costs and track usage. Genie can also help identify recently used models and create custom dashboards.
Key points
Section titled “Key points”- AI Gateway sits in front of models, agents, MCP servers, and tools.
- Access and spending can be managed in one control plane instead of separately for each tool.
- Multiple models, including Gemini, Anthropic, and OpenAI, can be placed behind one endpoint.
- Applications do not need to hardcode a model name.
- Users do not see which model is running underneath.
- New models can be added by updating the endpoint without changing the user experience.
- Requests can be distributed across models and tested in the playground.
- System tables log every gateway request and its cost.
- Genie can identify recently utilized models and build custom dashboards.
- Dashboards can show usage by model, user, and tool.
Chapters
Section titled “Chapters”| Start | Chapter | Summary |
|---|---|---|
| 00:00 | Why organizations need AI Gateway | AI Gateway centralizes access and spending controls for models, agents, MCP servers, and tools. |
| 00:31 | Creating an endpoint for multiple models | The presenter puts several models behind one endpoint so models can be tested or updated without changing the application. |
| 01:02 | Testing models and tracking costs | Requests are distributed across models, logged in system tables, and analyzed with Genie and dashboards. |
| 01:36 | A unified view of usage | The gateway helps teams see spending and usage by model, user, and tool while managing access centrally. |
Quotes
Section titled “Quotes”How much are we actually spending and on what? — 00:00
I don’t want to hardcode in my application a model name. — 00:31
Every request through the gateway gets logged into system tables. — 01:02
So, you can control access, track usage, and log what’s happening — 01:36
Concepts and entities
Section titled “Concepts and entities”Databricks, Unity AI Gateway, Agentyc AI Explain, Gemini, Anthropic, OpenAI, Genie, system tables
Full transcript (original language)
Section titled “Full transcript (original language)”00:00 The more AI tools we add, the harder it gets to answer one simple question. How much are we actually spending and on what? Welcome to Agentyc AI Explain series, where you build practical skills. This is why AI Gateway is becoming a must in every organization. It sits in front of your models, agents, MCP servers, and tools. So, you define access and spend once in one control plane instead of tool by tool. So, let me show you. I’m using Unity AI Gateway,
00:31 and I’m going to create an endpoint. Because models change every single week. So, I don’t want to hardcode in my application a model name. So, instead, I’m actually going to put several models behind one endpoint, and then I can test which one performs better. Gemini, Anthropic, OpenAI. For the user, this is completely seamless. They don’t see which model is running underneath. So, now when a new model drops, I can simply update the endpoint, and nothing changes
01:02 for them. So, now let’s test it in the playground. I will ask a simple question like a capital of a country, and I’ll test it the several times so that you can see that those requests are being distributed across those models. Now, this is the part that I actually care about. How much it cost? Every request through the gateway gets logged into system tables. So, I can see for every request how much it cost and compare it. So, here I’m asking Genie which models are utilized recently. I can also ask Genie to build a custom dashboard for my
01:36 team or my personal use. This allows you to move from guessing how much you’re spending across all these different tools to actually seeing it per model, per user, per tool in one dashboard. So, you can control access, track usage, and log what’s happening instead of managing everything separately across different tools. Now, try it yourself and I’ll see you in the next AI Explained video.