Two AI Models on One 8GB GPU. Zero Stutter? I Tested It

Author: Prompt Engineer 48 · Watch on YouTube

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: Two AI Models on One 8GB GPU. Zero Stutter? I Tested It · Channel: Prompt Engineer 48 · 05:41 · Watch on YouTube

Topics: Model inference and optimization, AI infrastructure and hardware, Local and open-source models, Multimodal models

Summary

The video tests a local voice assistant that runs a Gemma language model and a voice-cloning model on an 8GB RTX 4060 laptop. Running both models simultaneously caused audio gaps in all six test runs, while making them take turns eliminated stalls in all six runs. The turn-taking mode averaged zero seconds of audio gaps, but it did not improve time to first audio and made the language model take longer to finish writing. The test also found peak VRAM use of 7,407 MiB and noted limits around platform support, tool calling, and network security.

Key points

The tested project runs Gemma 4E4B and Quan 3TTS 1.7B on one 8GB GPU for a local voice assistant.
The project is described as an early-stage repository built for ESP32 voice boards.
When both models ran at once, all six tests had audio stalls, averaging 0.84 seconds of silence.
With the models taking turns, none of the six tests stalled; average gap time was zero.
Time to first audio was about 1.65 seconds in both modes, unlike the improvement reported against an older setup in the readme.
The language model took about 10 seconds to write while sharing the GPU and about 46 seconds when taking turns.
Peak VRAM use across the 12 runs was 7,407 MiB out of 8,188 MiB.
Answers had similar length and tone, but only one of six pairs was word-for-word identical, so the test does not establish that the text is unchanged.
The project has limitations: paste mode lacks tool calling, Linux CI checks only the C bridge, and the standalone server is intended for a trusted home network.

Chapters

Start	Chapter	Summary

Quotes

The only thing I change is who gets the GPU and when. — 00:00

That is the whole bug, not a bad model, just two models fighting over the same card. — 01:03

After six runs, zero of six had a stall. — 03:05

Peak VRAM was 7,407 MiB out of 8188. — 04:09

If you have an 8GB Nvidia card and want an offline voice assistant, this is worth a weekend. — 05:10

Concepts and entities

GPU Jet, Gemma 4 E4B, Qwen3-TTS 1.7B, RTX 4060, turns, VRAM, RunPod, Windows 11

Full transcript (original language)

00:00 Same laptop, same two AI models, same question. On the left, the voice keeps cutting out and on the right, it never stops once. The only thing I change is who gets the GPU and when. All right, in this video, I will run both versions on this 8GB GPU laptop and measure the pauses myself because one developer’s six runs is a good start, not proof. Let’s go ahead and test this out. I’m prompt engineer and I ran this on the same laptop the developer used. This is the GPU Jet repo, license Apache 2.0, latest release V 1.2. It puts a language

00:32 model Gemma 4E4B and a voice cloning model Quan 3TTS 1.7B on one 8GB card. The idea is a local voice assistant. The model writes an answer. The voice model reads it out in a voice cloned from a 3 to 10-second recording. Now, mind you, the repo is small, about five stars, and the author built it for ESP32 voice boards. So, treat this as an early project, not a finished product. The project itself says it was only tested on an RTX 4060 laptop with Windows 11. And that is exactly the machine I have

01:03 here. Here’s why this breaks. The speaker plays audio in a small buffer. The readme simulates half a second while the voice model is making sentence three. The language model is also generating on the same GPU. They slow each other down, so sentence three finishes late. The buffer runs dry and you hear a gap. In the developer’s own test, running both at once stalled in six out of six runs. That is the whole bug, not a bad model, just two models fighting over the same card. The fix is almost boring. After the language model finishes one sentence, it stops. The

01:33 voice model gets the whole GPU, makes that sentence, and only then does the language model write the next one. So, they take turns. Here’s what that costs and what it saves. According to the readme, time to first sound goes from 4.4 seconds to 2.3. Total gap time from 4.59 seconds to 0.09, and the two models together use about 7.3 GB out of eight. Those are the developer’s numbers on one machine, so I’m going to check them. First, can my laptop even run this? Terminal NVIDIA-SMI, and you can see it

02:03 is an RTX 4060 laptop with 81888 MiB and about 570 used by Windows before I start. The readme needs about 13 GB of disk and downloads 8.8. So, I run the setup script, it pulls the voice model, converts it to a GGUF, and downloads the Gemma file. While this is downloading, I will record my reference voice about 12 seconds and write down exactly what I said in a text file. Next, the order matters. I start the voice service first, then the language model. The

02:34 readme warns that if you do it the other way around, the voice model can get pushed into shared memory and run slower. Okay, both are loaded and VRAM now reads 7314 MB. Test one, both models at the same time. I ask the same long question every run. Tell me about your favorite season, make it long. The tool prints a number called underrun total. That is the seconds of silence a listener would hear. Run one, 1.01 seconds. Run two, 0.64. After six runs, six of six had a stall. The average gap

03:05 is 0.84 seconds and time to first audio averages 1.65 seconds. This is a simulated player with a half-second buffer, not my speakers. So, these gaps are what a listener would hear. Test two, same question but with the language model pausing after each sentence. This is the mode the repo calls turns. Run one gap, 0.0 seconds. Run two, not 0.0. After six runs, zero of six had a stall. Average gap, zero seconds. First audio, 1.67 seconds. So, on my machine, taking

03:38 turns changed the gap from 0.84 to zero. The stalls were short, 0.57 to 1.39 seconds in total per answer, but all six runs had them. And with turns, none did. What I did not reproduce is the faster first sound. It was about 1.65 seconds both ways. The read me is 4.4 to 2.3 compares against an older setup, not just this switch. And the prices time, the model finished writing in about 10 seconds when sharing and about 46 seconds when taking turns. Next memory I

04:09 watched Nvidia smitten during all 12 runs. Peak VRAM was 7,407 MiB out of 8188. That leaves 791 MiB free. The read me says the posture does not change what the model writes, only when so I compare the answers. The simultaneous answer was 170 words, the alternating 168. Same question, same seed, but only one of six pairs came out word-for-word identical because my llama server prefix cache reset returned an error, so I will not claim the text is unchanged. Length and

04:40 tone were the same. One more thing I checked, the voice. Each sentence is trimmed and leveled, but only from the first 0.6 seconds of the sentence. So if a sentence starts quiet and ends loud, you can hear the volume jump. I did not measure loudness, so I will not rate it. Now the honest part, straight from the read me. It was only tested on an RTX 4060 laptop with Windows 11. And on Linux, the CI only checks the C bridge. The paste mode does not support tool calling, so no device controls. The standalone server is for a trusted home

05:10 network only. The token is empty by default, so do not put it on the internet. If you have an 8GB Nvidia card and want an offline voice assistant, this is worth a weekend. If you need tool calling or a Mac or Linux box, wait. If you are looking for cheap and reliable GPUs, I use RunPod and the link is in the description. So in this video [clears throat] we saw two models share one 8GB card and on my laptop the total gap per answer went from 0.84 seconds to zero. Try it on your card and tell me your numbers. I’ll see you in another

05:41 one.