This speech-to-text model of just 16.9 MB .. crazy

Author: Prompt Engineer 48 · Watch on YouTube

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: This speech-to-text model of just 16.9 MB .. crazy · Channel: Prompt Engineer 48 · 03:48 · Watch on YouTube

Topics: Model inference and optimization, AI models, Local and open-source models, Multimodal models

Summary

The video tests Whistle, a 16.9 MB on-device speech-to-text model from Cactus Compute, running on a Windows laptop CPU without a GPU. The model supports seven languages and returns transcripts, language detection, word timestamps, and optional encoder embeddings. The presenter reviews the model card’s size, speed, and accuracy comparisons, then demonstrates installation and transcription, reporting a correct result in about a tenth of a second on a warm run. The video also shows word timestamps, keyword biasing, an empty result for silent audio, and deployment options listed on the model card.

Key points

Whistle is presented as a single 16.9 MB model file called whistle.kext, designed for on-device speech recognition.
It accepts 16 kHz mono audio up to 30 seconds in one pass and supports English, German, French, Spanish, Italian, Dutch, and Polish.
The model can return a transcript, detected language, word timestamps, and an encoder embedding.
The model card compares Whistle’s size and speed with Moonshine Tiny version two and Whisper Base; the speed figures were measured on an Apple M4 Pro, not the presenter’s laptop.
The reported word error rates vary by dataset, and the presenter notes that Whistle is not expected to beat larger models on noisy meeting audio.
Installation uses pip install cactus needle; microphone input or other sample rates require the mic extra.
On the presenter’s CPU laptop, the test sentence was transcribed correctly, and a warm run took about 0.11 seconds for a three-second clip.
Word timestamps include start and end times and a probability; the demo also shows keyword biasing and an empty transcript for a silent file.
The model card lists platform binaries, a static library, a header, a C API, and a WASI component as shipping options.

Chapters

Start	Chapter	Summary
00:00	Introducing Whistle	The presenter introduces Whistle as a 16.9 MB on-device speech recognition model and outlines its supported audio and output features.
01:08	Size, speed, and accuracy	The video compares Whistle’s size and performance with other models, reviews reported word error rates, and notes its limitations on noisy audio.
02:14	Installation and transcription test	The presenter installs the package, transcribes a test sentence on a CPU laptop, and reports the file size and warm-run time.
02:45	Timestamps, biasing, and deployment	The video demonstrates word-level timestamps, keyword biasing, and silent-audio behavior, then mentions deployment options and recaps the test.

Quotes

the whole model is one file, whistle.kext. — 00:00

that’s roughly eight and a half times smaller than Whisper Base. — 01:08

Turn off the kitchen lights and set a timer for 10 minutes. — 02:14

the warm run takes about 0.11 seconds for that 3-second clip. — 02:45

The output was an empty string. It didn’t invent text — 03:16

Concepts and entities

Whistle, Cactus Compute, Needle, Whisper Base, Moonshine Tiny version two, Apache License 2.0

Full transcript (original language)

00:00 Hey, this is Prompt Engineer. In this video we look at a speech-to-text model that is just 16.9 megabytes. Yes, megabytes. All right, this is Whistle from Cactus Compute and the whole model is one file, whistle.kext. You can see it right here in the file list. Now, mind you that everything I run in this video is on my own Windows laptop, CPU only, no GPU. Let’s go ahead and test this out. So, it’s an on-device speech recognition model and the card says it’s made for phones, wearables, robots, smart home, and even microcontrollers. It takes 16 kHz mono audio up to 30 seconds in one

00:33 pass. The output is the transcript, the language, the time of every word, and the encoder embedding if you want to do matching without decoding. It supports seven languages, English, German, French, Spanish, Italian, Dutch, and Polish. And the language is auto-detected unless you set it. Under the hood, it uses the same engine as Needle, the other model from Cactus, same.kext container, same quantization, same CPU kernels. The weights are two to four-bit and the license is Apache 2.0. Now, let’s look at their comparison against Whisper Base and Moonshine Tiny version two size. First, Whistle is 16.9

01:08 megabytes, Moonshine is 41.9, and Whisper Base is 145.3. So, that’s roughly eight and a half times smaller than Whisper Base. Time to first token on a 10-second clip, 11.1 milliseconds for Whistle, 22.8 for Moonshine, 73.2 for Whisper, and decode speed is 1,319 tokens per second for Whistle against 266 and 262. Mind you, these speed numbers are from their Apple M4 Pro, not from my laptop. For accuracy, the word error rate for Whistle is 4.31% on

01:40 LibriSpeech test-clean and 10.49 on test-other, 7.65 on SPG by speech, and 7.61 on TED-LIUM. It goes higher on the hard ones, 19.01 on earnings, 22 and 26.07 on AME. So, this is a good start for a 16.9 MB file, but it is not going to beat the big models on noisy meeting audio, and the card itself shows that. Setting it up is one command, pip install cactus needle. Base install is enough for a 16 kHz wave file. For other sample rates or the microphone, you need

02:14 the mic extra. Then the code is three lines, import needle, and needle.transcribe with the path of your clip. For the test, I made a clip with the Windows voice that says, “Turn off the kitchen lights and set a timer for 10 minutes.” And you can see that it gave me back, “Turn off the kitchen lights and set a timer for 10 minutes.” It also returns the language and the time to first token, 107 milliseconds on my laptop, and 278 tokens per second. The model downloaded itself the first time, and on disk, the file is 16,919,407

02:45 bytes. Now, if I run it again, the warm run takes about 0.11 seconds for that 3-second clip. That is the whole call on a normal laptop CPU. Next, word timestamps. I pass word timestamps equals true, and every word comes back with a start, an end, and a probability. If you look here, kitchen starts at 0.88 seconds, ends at 1.12, probability 0.99. The lowest one in this clip is 10 at 0.94, and it was written as a digit 10. There’s also keyword biasing. You pass a

03:16 list like cactus and needle, and it favors those spellings in the search. So, names and product words come out, right? My test clip doesn’t have those words, so it just returned the same sentence, and I’m not going to claim more than that. And I tried a silent wave file. The output was an empty string. It didn’t invent text, which is something the card promises, and I could confirm. If you want to ship it, the card says, “Each platform folder has a needle binary, a static library, and a header. There is a C API with needle load and needle transcribe, and there is a WASI component. You can also chain it

03:48 with needle. One command transcribes the clip, runs the text against your tools file and gives you one JSON. So, in this video we have seen a speech-to-text model that is a single 16.9 MB file, runs on the CPU, and transcribed my test clip correctly in about a tenth of a second. If you’re interested, you can go ahead and check this out on your own system as well, and let me know how it goes. The model is Whisper by Cactus Compute. Thanks for watching, and thanks to Cactus Compute for putting this out. I’ll see you in another one. Check out the next video.