NVIDIA Nemotron 3 Diarization Explained
Read the Portuguese translation →
Author: Prompt Engineering · Watch on YouTube
Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.
Original language: English
Original video: NVIDIA Nemotron 3 Diarization Explained · Channel: Prompt Engineering · 01:05 · Watch on YouTube
Topics: AI models, Multimodal models, Privacy and data protection
Summary
Section titled “Summary”NVIDIA’s NeMoTron 3-D diarization identifies up to eight speakers in real-time streaming audio, including when people talk over one another. It assigns speaker labels in the order voices first appear and remembers them when they speak again. The 100-million-parameter model works alongside speech-to-text models such as Parakeet, which transcribe the words while NeMoTron identifies who said them. The setup runs locally, so audio is not sent to the cloud.
Key points
Section titled “Key points”- NeMoTron 3-D can distinguish up to eight speakers in real-time streaming audio.
- It can identify speakers even when two people talk over each other.
- Speaker labels are assigned in the order each voice first appears, starting with speaker zero.
- The model remembers voices so returning speakers keep their original labels.
- NeMoTron 3-D is described as a 100-million-parameter model.
- It can work alongside speech-to-text models such as Parakeet.
- Parakeet transcribes the words, while NeMoTron identifies the speakers.
- The setup runs locally rather than streaming audio to the cloud.
Chapters
Section titled “Chapters”| Start | Chapter | Summary |
|---|---|---|
| 00:00 | Real-time speaker diarization | NeMoTron 3-D identifies up to eight speakers in streaming audio, including overlapping speech, and labels them in order of appearance. |
| 00:32 | Speaker memory and transcription | The model retains speaker identities and works with speech-to-text models such as Parakeet to pair words with speakers. |
| 01:05 | Where to find it | The video directs viewers to the link in its description to check out the model. |
Quotes
Section titled “Quotes”It can tell apart up to eight different speakers in real-time streaming audio. — 00:00
The first person to speak becomes speaker zero. — 00:18
And it keeps a small memory of every voice it has heard so far. — 00:32
And it runs entirely locally. — 00:48
Concepts and entities
Section titled “Concepts and entities”speaker diarization, NVIDIA Nemotron 3, Parakeet models, Prompt Engineering
Full transcript (original language)
Section titled “Full transcript (original language)”00:00 Nvidia just dropped NeMoTron 3-D diarization. It can tell apart up to eight different speakers in real-time streaming audio. So, if you’re taking notes in a meeting, you don’t just get what was said, you get who said it. Here’s how it works. As the audio streams in every fraction of a second, the model decides which of the eight speakers is talking, even when two people talk over each other. It labels voices in the order they first show up. The first person to speak becomes speaker zero. The next one is speaker
00:32 one, and so on. And it keeps a small memory of every voice it has heard so far. So, when speaker zero comes back later in the meeting, they are still speaker zero. It’s a tiny model, just 100 million parameter, and it works alongside the speech-to-text model you already use, like the Parakeet models. Now, in this setup, Parakeets write down the words, and NeMoTron tells you who said them. And it runs entirely locally. None of your audio gets streamed to some cloud where who knows what happens to it.
01:05 So, to check it out, link is going to be in the video description.