1M Parameter AI Pulls Speech Out of Music + Fan Noise

Author: Prompt Engineer 48 · Watch on YouTube

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: 1M Parameter AI Pulls Speech Out of Music + Fan Noise · Channel: Prompt Engineer 48 · 10:27 · Watch on YouTube

Topics: AI evaluation and benchmarks, Model inference and optimization, Local and open-source models, Multimodal models

Summary

The video tests Dynamic Band Splitting (DBS), an audio separation method that predicts frequency-band boundaries from each audio mixture, including the roughly 1-million-parameter DBS Tiger model. On a locally created 35.6-second mix of speech, music, and simulated fan noise, Tiger improved speech SI-SDR from 1.3 dB to 12.04 dB, while the larger DBS-BSRNN reached 11.19 dB. Tiger separated speech effectively, but neither model produced a clean, dedicated fan track, and the speech output retained some artifacts. The test also found Tiger slower than BSRNN, and the presenter emphasizes that results from one synthetic file are not a benchmark.

Key points

DBS predicts frequency-band boundaries from the audio while keeping the number of band tokens fixed.
The two released models discussed are DBS Tiger, with about 1 million parameters, and DBS-BSRNN, with about 4.7 million.
The presenter counted 1.01 million parameters for Tiger and 4.71 million for BSRNN, matching the reported model sizes.
The test mix contained clone-voice narration, music, and generated fan noise, and was 35.6 seconds long.
Speech SI-SDR rose from 1.3 dB in the mixture to 12.04 dB with Tiger and 11.19 dB with BSRNN.
Tiger scored 8.67 dB on music separation, compared with 2.94 dB for BSRNN; the presenter says BSRNN left more voice and fan noise in the music track.
Neither model isolated the fan into its own track; Tiger’s effects output scored 4.36 dB against the fan, while BSRNN scored -5.7 dB.
On the GPU, Tiger took 10.4 seconds and BSRNN took 5.1 seconds; peak VRAM use was about 1.2 GB and 1.05 GB, respectively.
The presenter cautions that the local test used only one synthetic file and one run per model, unlike the authors’ results over 973 test tracks.

Chapters

Start	Chapter	Summary
00:00	Introducing dynamic band splitting	The presenter introduces a speech, music, and fan-noise mixture and explains how DBS differs from fixed frequency-band separation.
03:12	Reported results and the listening demo	The video reviews the models’ reported DNRV2 scores and the browser demo, then prepares to test the models locally.
04:44	Installing and testing the models	The presenter installs DBS on an 8 GB RTX 4060 laptop, verifies parameter counts, and measures separation on a controlled mix.
07:18	Audio quality, speed, and caveats	The results show residual artifacts and different runtimes, while the presenter explains the limits of the single-file test.
09:24	Verdict and recommendations	The presenter recommends trying Tiger for extracting speech but not for isolating fan noise, and points viewers to the demo and code.

Quotes

And this is the speech that comes out of it from a model with only 1 million parameters. — 00:00

DBS predicts the band boundaries from the audio itself, but keeps the number of band tokens fixed. — 01:03

Before separation, the speech in the mix sits at 1.3 dB. — 05:47

I’d say it’s usable, but it’s not studio clean. — 07:18

This is not a benchmark. — 08:52

Concepts and entities

Dynamic Band Splitting (DBS), DBS-Tiger, DBS-BSRNN, Dynamic Band Splitting (DBS), DBS listening studio, Tsinghua, Bing Ja Zhang, Kai Li, Zhao Lin Hou

Full transcript (original language)

00:00 This is a mix of a voice and music track and a fan humming. And this is the speech that comes out of it from a model with only 1 million parameters. >> All right. So, this is a speech-to-text model and I gave it 8 and 1/2 minutes of >> All right. In this video, we will see dynamic band splitting from Tsinghua and you can see that the speech goes from 1.3 dB up to 12 dB. Mind you, this is running on my 8GB RTX 460 laptop. Let’s go ahead and test this out. Quick one. I am prompt engineer and I tested this so

00:33 you don’t have to. So, this is the GitHub repo, just release/dynamic band splitting. And the paper is called learning where to split, dynamic band splitting for audio source separation. If you look here, it’s from Tsinghua and the license is Apache 2.0. Now, one thing I want to say up front, if you look at the top, this repo has about 20 stars, zero forks, and it went public on October 10th. So, this is very new and nobody outside the authors has really tested it yet. That’s part of why I’m doing this. Here’s the idea. Models like

01:03 BSRNN split the frequency axis into fixed bands, same bands for every audio file. DBS predicts the band boundaries from the audio itself, but keeps the number of band tokens fixed. So, the cost stays the same and the model decides where to spend it. There are two released models, DBS-BSRNN with 4.7 million parameters and DBS-Tiger with about 1 million. And you can see in the news section, they also released a listening studio you can try in the browser. This is the paper page on Hugging Face. It has about 130 upvotes and the three authors are Bing

01:35 Ja Zhang, Kai Li, and Zhao Lin Hou. If you read the abstract, the claim is that DBS beats the corresponding fixed band models on three data sets, DNRV2, MUSDB18-HQ, and EchoSet. And this line here says DBS-BSRNN gets a 29% relative gain in average SISDR on DNRV2. And if you scroll down this project page, there is a card that says plus 29.0% the small text says DBS PSRNN goes from 7.3 to 9.07 dB over the reported BSRNN

02:09 result on DNRV2. So that 29% is 7.03 against 9.07. The next card says 128 band tokens held fixed. The boundary locations and widths move with the mixture and the token count does not. And a third card says four backbones improved BSRNN, Tigger, BS Reformer and Mel Reformer. And here is the method section of the project page. The scorer rates every frequency been a hard top K picks the boundaries and each band gets pulled into one token. Then the backbone

02:40 runs on those tokens and the result is expanded back to the full spectrogram. Further down there is a 30-second excerpt from the film Tears of Steel cut into 3-second windows and you can see the colored boundary lines sit in different places from one window to the next. The same paper is on archive as 2006.110.14993 version 1 listed as submitted on October 10th. Okay, so that is the claim. Now let’s look at the numbers they report and then I’ll check them on my own machine. This is the results table from

03:12 the readme on DNRV2. DBS Tiger gets 7.78 dB on music, 15.90 on speech and 6.70 on sound effects. DBS BSRNN gets 6.48, 15.01 and 5.73. So the model with 1 million parameters scores higher than the one with 4.7 million on every row averages 10.13 against 9.07. That’s the headline for this video. Now mind you DNR is a dataset of movie style audio so dialogue, music and effects. Those are

03:42 the three outputs the model gives you. A fan is not really one of those so I’m curious where it ends up. This is the hugging face space called DBS listening studio. It has only two likes so again, very early. You can upload an audio or a video file, and it gives you speech, music, and sound effects as separate tracks. If you look here, there’s also a view of the dynamic bands. This is the part that’s different from other separators. You can watch the boundaries move while the audio plays. And these are the two model cards, DBS Tiger DNR

04:14 and DBS BSRNN DNR. If you open the files tab, the weights are a small safe tensors file, so the download is quick and the hub shows around 176 downloads on the TB Tiger card. On the model cards, the Tiger weights file is 4.1 megabytes, and the BSRNN one is 18.9 megabytes. The hub shows about 176 downloads for Tiger and 91 for BSRNN. So, these are very new. I uploaded my 35-second mix to the demo with DBS BSRNN

04:44 selected, and it finished in about 8 seconds on their GPU with a speech, music, and effects tabs ready to play. The demo is great for a quick listen, but I want real numbers. So, let’s go ahead and install this on my laptop. Here’s my system, RTX 4060 laptop, 8 GB of VRAM, and everything you see here is running locally. The install is simple, get clone the repo, make a virtual environment, install a CUDA build of PyTorch first, and then pip install the requirements and the package with no deps. Python 3.11 or newer is needed,

05:16 and I’m on 3.13. One small thing, the end requirements file asked for torch 2.14.1, so I installed that and it worked with CUDA 13. Now, to check the model size claim, I loaded both models and counted the parameters. You can see DBS Tiger prints 1.01 million and DBS BSRNN prints 4.71 million. So, the numbers on the page match. Here is the test. I’m going to be honest about this. I did not record in my room. I built a mix of 35.6 seconds with my clone voice

05:47 narration, a music track, and a generated fan, which is band-limited noise plus a hum at 120 and 240 Hz. The reason is that I know exactly what is inside, so I can measure the result with SISDR and not just trust my ears. Before separation, the speech in the mix sits at 1.3 dB. That’s very messy. Let’s go ahead and run DBS Tiger with a 6-second windows from the read me, and you can see the speech comes out at 12.04 dB. Music is 8.67 dB.

06:17 >> All right. So, this is a speech-to-text model, and I gave it 8 and 1/2 minutes of audio in one go. The normal way of running with >> Now, DBS RNN with the 3-second windows, speech is 11.19 dB. So, both give about a 10-dB improvement on speech, and Tiger is almost a full dB better with a quarter of the parameters. >> All right. So, this is a speech-to-text model, and I gave it 8 and 1/2 minutes of audio in one go. >> The normal music is where they split. Tiger gets 8.67, BSRNN gets only 2.94,

06:48 so BSRNN left a lot of the voice and fan in the music track. >> All right. So, this is a speech-to-text model, and >> And now the fan, I compared the sound effects track to the fan. Tiger gets 4.36 dB, so a good part of the fan landed in the effects track. BSRNN gets -5.7 dB, which means the fan is mostly spread across the other stems, so the fan does not have its own track in

07:18 either model. If I listen to the Tiger speech, the music is gone, the fan is mostly gone, and the voice has a slightly thinner, processed sound. I’d say it’s usable, but it’s not studio clean. >> All right. So, this is a speech-to-text model, and I gave it 8 and 1/2 All right. So, this is a speech-to-text model, and I gave it 8 and 1/2 minutes of audio and one. >> If you look at the speech spectrogram, you can still see thin vertical streaks reaching up high. Those are hits from

07:50 the music track that leaked into the speech, so it is not perfectly clean. Okay, speed. The file is 35.6 seconds long. On the GPU, DBS Tiger took 10.4 seconds, and DBS BSRNN took 5.1 seconds. So, the small model is actually slower. That surprised me. I think it’s because Tiger uses 6-second windows and has a heavier structure on those tokens. So, it runs more compute even with fewer parameters. That’s my guess. I did not profile it. Peak VRAM was about 1.2 GB for Tiger and

08:21 1.05 for BSRNN. So, even a 4 GB card would be fine. And on the CPU, BSRNN took 71 seconds, and Tiger took 196 seconds. So, a GPU matters. Tiger on the CPU runs about 5.5 times slower than real time. The numbers I show are after one warm-up pass, so the first run on your machine will be a bit slower. So, here’s what I did not test. It is a synthetic mix, one file, 35 seconds, one run each. This is not a benchmark. The author’s numbers are over 973 test

08:52 tracks. Mine is one file. The models were trained on DNR V2, which is movie-style audio. A fan is outside that, and you saw it did not get its own track. Also, I only tested the DNR models. The MUSDB versions with vocals, drums, and bass are a different setup. And the repo has about 20 stars. The authors have not published user reports, so treat these results as a good start. And if you try it on a real room recording, let me know how it goes. One more thing, the demo runs on a shared A10G 0 GPU space, so the 8 seconds I saw

09:24 includes their upload and queue, and it is not comparable to my local timings. My verdict: For pulling speech out of a mix, DBS Tiger did what it says, 1.3 dB to 12 dB in my test at 1 million parameters and about 10 seconds on an 8 GB’s laptop for podcast cleanup or extracting dialogue from a video with music, it’s worth a try. If you need a clean fan or noise track, it will not do that. And if you want the fastest, BSRNN was twice as fast here, but the speech was about 0.85 dB worse and the music was much worse. If you just want to

09:56 listen, the hugging face demo is enough. If you want a pipeline, the Python snippet in the readme is only about 10 lines. If you’re looking for cheap and reliable GPUs, I use RunPod and the link is in the description. So, in this video we have seen how dynamic band splitting pulls speech out of a mix on a local 8 GB GPU. The repo, the space, and the paper are all in the description, so you can go ahead and try this out on your own recordings as well and let me know how it goes. I’ll see you in another one. Check out the next video. If this helped, subscribe and next I

10:27 will test another tiny model on this same laptop.