Audio8 ASR Infinite: 89% Wrong vs 6% Wrong, Same Audio

Author: Prompt Engineer 48 · Watch on YouTube

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: Audio8 ASR Infinite: 89% Wrong vs 6% Wrong, Same Audio · Channel: Prompt Engineer 48 · 05:45 · Watch on YouTube

Topics: AI evaluation and benchmarks, Model inference and optimization, AI infrastructure and hardware, AI models, Multimodal models

Summary

The video tests Audio8 ASR Infinite, a speech-to-text model, on a laptop and a rented RTX 4090, then compares its standard PyTorch and vLLM inference paths. The 8 GB laptop cannot load the model, and the PyTorch path fails on long audio after about two minutes, reaching 89% errors. With vLLM, the same 8.5-minute recording gets 6% errors, and a 34-minute continuous test maintains roughly consistent accuracy and memory use. The model handles clean English and Chinese well, but struggles with noise, mixed-language numbers, names, and jargon.

Key points

Audio8 ASR Infinite is described as a 4-billion-parameter Apache 2.0 model for English and Chinese, with a 30-second checkpoint context and a rolling cache intended to sustain long streams.
The model would not load on the test laptop with an RTX 4060 and 8 GB of VRAM.
On an RTX 4090, a known sentence was transcribed with zero errors, while scoring number formatting as digits initially inflated the error rate.
A Chinese sample had zero errors, but a mixed Chinese-English sample containing “RTX 4090” reached 20% character errors.
Noise, accelerated audio, and phone-quality audio reduced accuracy; the reviewer identifies noise as a weak spot.
Four of six LibreSpeech clips scored under 6% errors; the video notes the model card reports 3.04% on the full clean set.
In plain PyTorch, the 8.5-minute recording deteriorated after about two minutes and reached 89% errors.
With vLLM, the same recording scored 6% errors; a 34-minute repeated stream scored 5.4% overall with largely stable step time and memory.
The reviewer says longer delays can improve accuracy, with diminishing returns, and observes an end-of-turn signal during silence.
The video’s conclusion highlights punctuation, capitalization, and digit formatting as strengths, and version pinning, laptop requirements, and jargon errors as drawbacks.

Chapters

Start	Chapter	Summary
00:00	Model overview and test plan	The reviewer introduces Audio8 ASR Infinite, its streaming design, and a plan to test long-form transcription.
01:05	Setup and short-audio accuracy	The laptop fails to load the model, so the reviewer moves to an RTX 4090 and tests clean speech, Chinese, mixed-language audio, and noisy clips.
02:38	Real speech and the long-audio failure	The reviewer reports results on LibreSpeech and finds that plain PyTorch collapses after about two minutes of an 8.5-minute recording.
03:41	vLLM long-stream tests and settings	With vLLM, the same recording and a 34-minute repeated stream maintain low error rates, while the reviewer also tests delay and end-of-turn detection.
04:43	Cost, final sample, and verdict	The reviewer estimates the cost of a continuous stream, tests narration from the video, and summarizes the model’s strengths and weaknesses.

Quotes

The checkpoint only has a 30-second context, but a rolling cash is supposed to keep memory and latency flat forever. — 00:32

It just writes the the the 89% errors from there on. — 03:10

Same 8 and 1/2 minutes, same audio. Now, it’s 6% and no minute goes over eight. — 03:41

I only tested one stream, so a 24/7 stream on this pod is about $17.76 a day. — 04:43

So, the good, Apache 2.0. It puts in punctuation, capitals, and digits for you. — 05:14

Concepts and entities

Audio8 ASR Infinite, Edge Zero, vLLM, PyTorch, RunPod, RTX 4090, LibriSpeech, Apache License 2.0

Full transcript (original language)

00:00 All right. So, this is a speech to text model, and I gave it 8 and 1/2 minutes of audio in one go. The normal way of running it fell apart after 2 minutes. The other way got it down to 6% errors. Let’s go ahead and see why. It’s called audio 8 ASR infinite from a team called Edge Zero. It’s a 4 billion parameter model Apache 2.0 English and Chinese. It listens as a stream 12 and 1/2 times a second and writes one token every time. Under the hood, the ears are the Vox Troll real-time audio tower, and the mouth is when 2.53 billion. The model

00:32 file is 8.17 GB in BFloat16, so keep that number in your head. Now, the big claim on the card is 24/7. The checkpoint only has a 30-second context, but a rolling cash is supposed to keep memory and latency flat forever. That is the thing I want to check. So, what I’ve done is I’ve made a plan. One, try it on my own laptop. Two, if that fails, rent a GPU. And three, throw a lot of different audio at it and see what breaks. First, my laptop. This is an RTX 4060 with 8 GB. The weights are 7.6 GB,

01:05 and Windows already holds about 1 GB of the card. So, free memory here is 6.93. Even so, let’s go ahead and try. The load crashes with an access violation twice. Then I load straight to the GPU, and the OS says the paging file is too small. So, it’s a no on this laptop. Let’s rent a GPU. So, I rented one RTX 4090 on RunPod, 24 GB, 74 cents an hour. I installed torch 2.13 and transformers 5.13, the exact versions the repo pins.

01:36 One thing broke, the old tor git tor git portion in the image. I removed it, and it worked. First test, I made a sentence with my own cloned voice, so I know the exact words, and you can see it got every word right with the commas and the capital letters. Zero errors. Next, numbers and jargon. I said 24 GB, and it wrote 24 with digits. My scorer counted that as a mistake. 17.9% after I fixed the scorer, it’s 3.6. I’m showing you both numbers. Now, Chinese this sentence

02:08 zero errors. Then I mixed Chinese and English with a number in the middle and this one falls over. I said RTX 4090 and it wrote something completely different. That’s 20% character errors. Now, let’s make it hard. Pink noise on top and the end of the sentence breaks it. Writes 74 an hour on one part. Speed it up 1.75 times 10%. Phone quality audio 5%. So, noise is the weak spot here. Now, real people, these are six clips from LibreSpeech. Four of the six are under

02:38 6%. The longest one, a 30-second clip, is 14 and the worst is a short clip where it heard Norris instead of Norris. Their card says 3.04% on the full clean set. So, I’m not going to claim anything against that. How fast is it? In plain PyTorch on the 4090 between 0.8 and 0.9 of real time. So, it keeps up, but it’s not fast. Peak memory is about 8 GB, so it fits on a 24 GB card with a lot of room left. Now, the important has 8 and 1/2 minutes of real speech in plain PyTorch. The first

03:10 minute is fine, 7% errors. Then after about 2 minutes, it collapses. It just writes the the the 89% errors from there on. So, the plain PyTorch path is not the infinite one. The infinite part lives in vLLM. The repo’s official way is Docker, but a pod can’t run Docker, so I installed vLLM 0.27.1 directly and set the plugin by hand. And these are the three health lines that tell you it’s injected, the rolling scheduler, the real-time endpoint and only one real-time road. Same 8 and 1/2 minutes,

03:41 same audio. Now, it’s 6% and no minute goes over eight. 1150 words in, 1150 words out and the GPU memory 10,428 to 10,438 MB the whole time. Okay, so 8 minutes is not 24/7, so I looped the same audio four times, 34 minutes in one continuous stream. Every 2 minutes stays between 3 and 6 and 1/2 percent. 5.4 overall, the step time. First minute, 57 milliseconds. Last minute, 58. Memory sits at 10,487 with one spike of about 400 megabytes.

04:13 You can also trade delay for accuracy. On 2 minutes of real speech, 240 milliseconds gives 8. 2%, 327, 480, 6.7, 566.3. So, every extra bit of waiting buys you less and less. There’s also a semantic end-of-turn signal. I made a clip with a 3-second silence in the middle and 5 seconds at the end. In the gap, the probability averages 0.8 and peaks at 0.95. While I’m talking, it’s around 0.1 with one bump to 0.64 at a comma. Honest note though, plain silence

04:43 is the easy case. I did not test a thinking pause. One more thing, cost on paste. It took 400 51 seconds to do 510 seconds of audio. So, a bit faster than real time, one stream on 14090. I only tested one stream, so a 24/7 stream on this pod is about $17.76 a day. And one last test, and this is my favorite. I took most of the narration of this very video and gave it to the model. It got it down to 6% and the mistakes are the fun ones. It heard Vox Stroll as Vox Stroll, Quan as Kean, and

05:14 Tor Giap Show as Tor Giap Tor Giap Portion. So, names and jargon are where it slips. So, the good, Apache 2.0. It puts in punctuation, capitals, and digits for you. Chinese was clean and the rolling window really works in VLLM. The bad, it will not run on an 8-gig laptop. The plain PyTorch path breaks after 2 minutes. Mixed language with numbers failed. And it’s pinned to exact versions. So, that’s it. Everything I showed is real output from a real run. Next, I want to try their web client with a live microphone and see how many

05:45 streams one card can hold. I will see you in the next one. Thank you.