VoiceStudio: Local Voice AI That Doesn't Beat ElevenLabs

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: VoiceStudio: Local Voice AI That Doesn’t Beat ElevenLabs · Channel: Fahd Mirza · 12:35 · Watch on YouTube

Topics: Local and open-source models, Multimodal models, Open source and software ecosystems

Summary

The video introduces Voice Studio, an open-source local voice toolkit, and walks through installing it with Docker and configuring its interface. The presenter emphasizes that Voice Studio is a container for different engines and argues that its default models are weak, demonstrating how to install and select VoxCPM 2 instead. He tours features including voice cloning and design, dubbing, narration, audiobooks, a voice gallery, transcription, and dictation. The conclusion is that Voice Studio is still clunky and is not a replacement for ElevenLabs, despite the hype.

Key points

Voice Studio is an open-source toolkit that runs locally and offers voice cloning, voice design, dubbing, dictation, transcription, and audiobook creation.
It supports a local API and MCP, can run as a desktop app or in Docker, and its models may have separate licenses.
The presenter installs it in Docker with GPU access, creates an API key, and checks that the system is ready.
The available engines and languages depend on the selected model; the presenter criticizes the default models’ quality.
VoxCPM 2 is installed manually inside the running container because the catalog installation did not work for the presenter.
The voice cloning demo uses an uploaded sample, and the presenter notes that emotional controls depend on model support.
The tool also supports audio or video dubbing, with video models potentially requiring around 80 GB of VRAM.
Story and audiobook features let users assign voices to characters and generate narration locally.
The presenter describes the interface as clunky and says Voice Studio does not match ElevenLabs as a drop-in replacement.

Chapters

Start	Chapter	Summary
00:01	Voice Studio overview and Docker installation	The presenter introduces the toolkit, notes licensing and voice-cloning cautions, and begins installing it locally with Docker.
02:03	First launch and setup	He connects to the web UI, checks the system, selects models, and completes the initial configuration.
04:09	Model catalog and VoxCPM 2	The presenter explains why he recommends alternatives to the default engines and demonstrates manually installing and selecting VoxCPM 2.
06:16	Voice cloning demonstration	He selects VoxCPM 2, uploads a voice sample, and demonstrates synthesis while noting that results and emotions depend on the model.
08:55	Dubbing, stories, and dictation	The presenter tours dubbing, character-based story editing, the voice gallery, and live dictation.
11:32	Audiobook generation and final assessment	He plays a generated audiobook and concludes that Voice Studio is clunky and not a replacement for ElevenLabs.

Quotes

It is a voice toolkit that runs locally on your own hardware. — 00:01

I would highly, highly recommend that for any task like voice clone, voice design, voice dubbing, even dictation, you go with a better models — 04:09

It depends on if the model supports the emotions or not. — 08:24

this is not a drop in replacement to 11 Labs by any stretch of imagination. — 12:04

Concepts and entities

Voice Studio, AGPL-3, Docker, MCP, OmniVoice, K2FSA, VoxCPM 2, Whisper tiny, ElevenLabs, Fahd Mirza

Full transcript (original language)

00:01 Today, we are looking at Voice Studio, an open-source project on GitHub under the AGPL 3 license. It is a voice toolkit that runs locally on your own hardware. It covers voice cloning, voice design, video dubbing, dictation, transcription, and audiobook creation. The readme says it supports 646 languages. By default, it uses OmniVoice from K2FSA as its engine, and you can

00:32 switch to other engines from the catalog. It also has got a local API and MCP support, so agents can call it. You can run it as a desktop app or with Docker. Note that models have their own licenses, so check them before commercial use, and clone voices only with permission. As you can see that I’m using my Ubuntu system. I have one GPU card, which is Nvidia RTX with 48 GB of VRAM. If you’re looking to rent a GPU on very good price, you can find the link to my Mask

01:02 Computer videos description, and you can subscribe to my weekly AI newsletter at wahabmirza.substack.com. And if you want to help out the channel, please become a member. You know what? Let’s go straight to terminal and get this thing installed. I’m going to use the easiest way of Docker. Before that, let me quickly create this local token. So, this is just a random API key that protects the web UI, and I’m just going to use it

01:33 once, so don’t worry about it. But, if you’re using it in production, make sure to store it somewhere safe. Next up, let’s start the Voice Studio container with GPU access, saving data and model cache on the VM. So, it is going to download it, and then going to launch on our local system. So, let’s wait for it to get downloaded. Everything is downloaded. You can also quickly do the health check. And it has returned okay, so we are

02:03 good. Let’s now open it in the browser. And first time when you run this, it asks you for the API key. That is the same random token which we generated. So, I have just um used it. So, let me click click on connect. And there you go. So, uh it is connected. And it is asking me, you know, what sort of UI I want. I’ll just go with the suggested one for now. So, let’s use 80%.

02:35 It is loading. Probing the system. Let’s wait. So, you see everything checks out. My disk space, my GPU, network, runtime. So, and it is also telling me all good continue. So, let’s click on this pink button. Next, it is asking me which model would I like. So, you can just scroll through it. There are heaps of models available. I’m just going to go with a default, but you can see that you can

03:06 uh go with pocket, uh index, and lot of other good ones. And then you can also, you know, put up the hugging face token here. So, let me paste my token. And then click on um required models. Once that’s done, uh because the first one was Studio TTS is required. Let’s click on download. And then if you come just down here, so this is another one which is already active. So, it is ready. So, let’s continue.

03:38 And then let’s not do it for now. I’m already doing it here. So, the feedback is this video. Okay, um now again, I think I’ll just go with uh you know, whisper tiny. Though, I really like the parakeet one, but I’m just going to go with Whisper for now. And then it is also, you know, checking out um you know, what sort of vocabulary do I want? I’ll just go and go with Enter Studio.

04:09 Okay, now let’s test it out. This is the launchpad of this voice studio tool. Now, one thing which you need to make a very clear is that this is just a tool, a container, where they have their uh sort of default models plus lot of other engines. I would highly, highly recommend that for any task like voice clone, voice design, voice dubbing, even dictation, you go with a better models instead of the models they have put in as default because they are not good at all. And I will show you. So, at uh the

04:42 on the left-hand side at the bottom, you see there is catalog. Click on it. This is a model catalog, where speech is this Kitten TTS or if you change it here, they come in come uh with their voice studio uh Omni Voice model, which is not good at all. Transcription is again Fast Whisper and Whisper Tiny. So, what I would suggest that these are all the engines, you go with a better ones. We have covered each and every one of them. So, for instance, if I click on Vox CPM, this one is quite good. It has got

05:13 around 30 languages, voice design is there. Now, unfortunately, you cannot just go and install it from here. So, the weights are not there. So, what I’m going to do, I’m just going to delete it from here because I tried installing it, it doesn’t work from here. Otherwise, I could just click install and go from there. So, for this, you would need to go and follow this guide. Now, if you read through this guide, it is quite poorly written because it will be very hard uh to get it installed this way. So, let me

05:45 take you back to my terminal and show you how to get this thing installed, the Vox C PM. My Docker container is already running, so I’m just going to install that Vox C PM package, as you can see, inside this container. And it is going to take a couple of minutes, but will get installed. It is successfully installed, and then you just need to restart your container. That is done. Let’s go back to our model

06:16 catalog, and from there, let’s go here and just um you need to reload and then put in your API key. Let’s Let me do that. And now, if you scroll down, you will see that these are engines which are ready to use. And Vox C PM 2 is one of them, so I’m just going to click on use here. And now, I have switched my TTS to Vox C PM 2 with 30 languages. On the left-hand side, you see there are a lot of options. So, there is this one voice. I’ll click here.

06:48 This is where you can do the voice cloning. Uh if you want to see um what I you know, what is the quality of this one, you see um the Omni voice, the default one, it’s very poor. So, I’m just going to play it. >> Private, local, and ready whenever you are. Height my and yet it he when phone pre it come new >> But this is model, not the tool, and this is the Omni voice one. That is why I would highly recommend you go with this TTS. Anyway, so what I’m going to

07:21 do, this is asking about voice clone. At the top, just click on audio. You can clone it from audio. You can design a voice, and you can convert it. I’m just going to go back to the audio. I am going to upload again, maybe a voice. I’ll just go with my own voice, because that is what I normally test on. So, this is a paragraph. This is a voice. Let’s click on synthesize audio. I’ll play my original voice here.

07:52 Joy is found in simple moments of gratitude, and true contentment comes when we truly value the small black So, this is audio. Let’s see what it does here. I’ll play it out. But again, as I said, let’s not focus on models too much. I just want to demo this tool, how exactly this works, what are the selling features, and how to get it installed so that you have starting point, and then you can go further and explore explore yourself. The cloning is done. Let me play it out

08:24 for you. Hi, it’s Fahad. Subscribe to my newsletter at fahadmirza.substack.com. It is weekly. It is about AI. It is free, and there is no spam. I will keep saying this in every video until you subscribe. Tomorrow, I will say it again. Hi, next week I will say it louder. In a month, my So, you see, it depends on if the model supports the emotions or not. You can insert various emotions and other things from here.

08:55 Okay, and as I said earlier, you can also do by design and by convert. Let’s go to the next one. On the left, you can do the dubbing, but you can drop audio or video. The only thing you need to be aware of is that for video, most of the models would need around 80 GB of VRAM. So, next one is that you can also go with some of the stories editor. This is actually quite interesting, where you see they have this narrator, and then you can just select any model. Um

09:28 I would again say just go with the you know, walk CPU or any better one. And then you can just build this narration. You can also include more characters like, you know, for example, you can subs- uh subtract the character. So, for example, you can just uh call it Alice or whatever, and then default just you know, from the gallery, just select anything of your choice, and same with the other ones.

09:58 And then you can just keep writing your script. You can add lines or you can add chapters. Then you can also create audiobooks. So, this is a script, voices, and then this is a whole book which you can read through. And now, let me quickly show you the gallery. So, this is a gallery where they have different voices and stuff. So, you see a lot of things are there. Community is also adding terms if you’re interested in various languages, you can also check it out. And the language, by

10:29 the way, is also depends on the model you are selecting. And then this is a transcription, which you can start the dictation. So, for example, I’m not sure if my mic is connected. I just click on dictation. I will allow my mic. This is Fahad Mirza, and I welcome you to the channel. So, you see, you can also do the dictation, and then you can play it

11:01 back. So, I just stop it. Again, don’t worry about the quality of the dictation because, as I said earlier, it depends on the model. Now, in order to create an audiobook, all you need to do is to write a script just like this one, uh like a chapter heading, a pause, or a voice name for each character, and Voice Studio turns it into uh a narration audiobook with different voices all generated locally on this machine. So, I just clicked on the top generate audiobook, and it is just going

11:32 through the chapter and it is going to create that book for us. So, let’s wait for it to get generated and then we will check it out. And the story is created. Let me now run this. >> The storm had been building for 3 days and Mara knew, the way you know your own heartbeat, that tonight the old lamp would be tested. >> She climbed the spiral stair, counting each step out of habit. >> 101 102 >> At the top, the wind found her.

12:04 >> You flickered for 40 years and you’ll hold for one more. >> The lamp answered with a cough of light, then steadied. >> So, you see, um and you can already feel that the tool is still quite clunky. Uh it is not as good as 11 Labs, by the way. Because if you look go to the social media, there is a lot of hype around it, but you know, I feel that this is not a drop in replacement to 11 Labs by any stretch of imagination. But, let me know your

12:35 thoughts in the comments. As always keen to know your thoughts. Again, if you want to help out the channel, please become a member. Thank you for all the support.