The Tiny TTS Everyone Loved Isn't Tiny Anymore (KittenTTS 2)

Author: Better Stack · Watch on YouTube

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: The Tiny TTS Everyone Loved Isn’t Tiny Anymore (KittenTTS 2) · Channel: Better Stack · 07:49 · Watch on YouTube

Topics: AI evaluation and benchmarks, AI governance and regulation, Local and open-source models, Multimodal models

Summary

KittenTTS 2 adds voice cloning from a short audio sample and more capable speech generation, but it is a major departure from the original tiny, Apache-licensed models. Its roughly 1.7-billion-parameter model is much larger, can run slower than real time on a Mac, and relies on a separate vocoder and other dependencies. The model weights use a restrictive community license with commercial conditions, while quality claims have not yet been independently verified. The video considers it promising for experimentation, but cautions that its size, licensing, and early state may make it a poor fit for production.

Key points

The original KittenTTS models were tens of megabytes and licensed under Apache 2.0; KittenTTS 2 is a substantially larger and different kind of model.
Voice generation uses two stages: a speech language model creates audio tokens, and a vocoder turns them into sound.
KittenTTS 2 uses a new 1.7-billion-parameter model and the S3GEN vocoder from Resemble AI’s Chatterbox Turbo.
The default model build is about 950 MB, with smaller and larger versions also available, and 47 built-in voices.
The model can clone a voice from a short recording, but on a Mac it falls back to CPU rather than using the Apple GPU; CPU generation is documented as roughly two to three times slower than real time.
Local use has privacy and cost advantages: there is no account or ongoing service fee, and the voice recording stays on the user’s machine.
The Stellan Labs Community License applies to the model weights. Commercial users must register, and the free commercial grant ends if they exceed the stated revenue or funding threshold.
Reported speaker-similarity scores come from the project; the video notes that version 2 had not yet been independently tested.
The package brings substantial dependencies and model files, and the project was changing rapidly at the time of the video.
The host sees KittenTTS 2 as worth trying for an MVP or experimentation, but suggests production builders consider alternatives.

Chapters

Start	Chapter	Summary
00:00	How KittenTTS 2 differs from the original	The video introduces the new voice-cloning capability and explains how its size, performance, and openness differ from the original.
01:03	Model architecture and a voice-cloning demo	The host describes the language model and vocoder, then installs KittenTTS 2 and tests voice cloning and emotion tags.
03:07	Mac performance and licensing trade-offs	The video covers CPU-only performance on Mac, local-use benefits, reported quality scores, and the model weights’ community license.
05:43	Limitations, dependencies, and project maturity	The host reviews language and text-generation caveats, questions unverified CPU performance claims, and notes the heavy, rapidly changing installation.
07:18	Who should use KittenTTS 2?	The video concludes that it may suit experimentation, while production users may prefer alternatives because the project is larger and its license is more complicated.

Quotes

The original models were tens of megabytes Apache 2.0 license and small enough to run pretty much anywhere. — 00:00

Their own docs say CPU generation runs roughly two to three times slower than real time. — 03:07

The model weights are not. So, Kitty TTS 2 is source available. It’s free, but it’s it’s not open source. — 05:43

Kitten TTS did not just get bigger. The whole meaning of this project just completely changed. — 07:49

Concepts and entities

KittenTTS 2, S3GEN, Chatterbox Turbo, Resemble AI, Stellan Labs Community License, Apache License 2.0, ElevenLabs, Whisper, Hugging Face, Better Stack

Full transcript (original language)

00:00 Kittens TTS got popular because it was tiny. The original models were tens of megabytes Apache 2.0 license and small enough to run pretty much anywhere. Now there’s Kittens TTS 2 and it can clone a voice from just 5 seconds of audio but it’s also about 1.7 billion parameters. It runs slower than real time on a Mac and the model weights are no longer open source. So this is a very different Kittens TTS from the one we originally got. I installed it, tried the voice cloning and looked at how it actually

00:31 runs locally. Let’s take a look. Now a lot of people really did love the original but to understand why version 2 became so different we first need to understand how the voice cloning works. There’s basically two stages. A speech language model takes your text plus a short recording of a voice and it generates audio tokens. Then a second model called a vocoder turns those tokens into actual sound. Think of it

01:03 like this. The language model writes the sheet music, the vocoder is the orchestra that actually plays it. They got to work together, right? Two independent pieces. And this is where Kittens TTS 2 starts changing. The composer is brand new. It’s a 1.7 billion parameter model but the orchestra isn’t theirs. The default vocoder is S3GEN from Resemble AI’s Chatterbox Turbo and Kittens downloads it the first time you run the model. So we’re already pretty far away from the

01:33 tiny standalone model Kittens used to be. Now to shrink the main model down the weights are ternary at about 1 and 1/2 bits each. The default build is around 950 megabytes. There’s also a smaller version which is around 470 and a full version at around 3 and 1/2 gigs and you get 47 built-in voices but voice cloning is obviously the feature people are going to care about. So, let’s actually try it. If you enjoy coding tools to speed up your workflow, be sure to subscribe. We have

02:04 videos coming out all the time. Installation is just pip install kitten ML on Python 3.10 or newer. Then I record 10 seconds of myself talking. This is a test. I’m just talking to see how this sounds. 1 2 3 4 5. How’s this voice audio? Kitten even runs Whisper to transcribe the reference clip for you. Then I pass that clip in, generate the audio, and here’s me saying something I never actually said. This is a sentence I never actually recorded. If this

02:35 sounds pretty decent, you should subscribe to the Better Stack channel. Okay, what did we think about that? You can also add emotion tags and sounds like a laugh. Those are still marked beta though, and the voice cloning is really cool, but here let’s try it. We actually won. Can you believe it? >> [laughter] >> I can hardly believe it. Yeah, okay, it’s not perfect. But there’s still a pretty big catch if you’re on a Mac. If I fire up Activity Monitor, the CPU is pinned. The GPU is doing basically

03:07 nothing. And that’s because the code checks for an Nvidia card. If it doesn’t find one, it falls back to CPU. So, on a Mac, your Apple GPU isn’t being used at all. And their own docs say CPU generation runs roughly two to three times slower than real time. Their one recommendation for speeding it up is setting torch to use eight threads. It helps. It still doesn’t make it real time. So, at this point, if it’s not completely open source, we’re wondering why use this instead of something like Eleven Labs. And this is where Kitten

03:38 still has a real advantage. Eleven Labs starter, we’re paying a few bucks a month, and that gives you instant voice cloning and a commercial license. Kitten runs locally. There’s no permanent cost. There’s no account I have to create, and your voice, which we like, never leaves that machine. For some of us, that is a really good trade. But, there’s something much bigger you need to know before building anything with it. This is not the Kitten TTS we remember. The old versions were between 15 and 80 million parameters, around 25 to 80

04:10 megabytes with no voice cloning. It was still though Apache 2.0. Kitten TTS 2 is basically a complete different class of model wearing the same name. And that difference isn’t just technical. It also changes what you’re actually allowed to do with it. But, before we get to the license, let’s just talk about quality, which I’m guessing a lot of you guys just heard. They report speaker similarity score between .49 and .72 and .81 when cloning a recording it hasn’t

04:43 seen before. Those are their numbers. Nobody independently has tested version 2 yet. It’s a few days old and has around 130 downloads on Hugging Face. So, right now you’re mostly taking the project’s own benchmarks at face value. We got to test them. And now we get to the part I’d read before shipping anything with this, which is that license. It’s called the Stellan Labs Community License. Research use, non-commercial use, and commercial use are all royalty-free, which sounds great until we keep reading. The grant is

05:13 revocable. Great. Okay. Not really. And if you’re using Kitten commercially, you have to register with them by contacting them. If your company or you makes more than a million dollars in revenue, awesome. Or it has raised more than a million dollars in total funding ever, the free commercial grant completely ends. So, if you’re a startup that raised a seed round, you’re already cut out. Then, there’s more. You have to prominently show powered by Stellan Labs. You have to ship the license with a notice file. You can’t use the model

05:43 to train or distill other models. And there’s an acceptable use policy. Except that policy isn’t public. You have to contact them to read it, which is weird. Now, to be fair, the code itself and pip package are Apache 2.0. The model weights are not. So, Kitty TTS 2 is source available. It’s free, but it’s it’s not open source. There are a few other quality things worth knowing before you use it. For non-English voices, each one relies on a single reference clip. And their own docs

06:14 describe those voices as more fragile. The docs say 10 languages, but the readme says 20. And when generating non-English text, you need to turn normalization off. Long text also gets split into chunks automatically because the model can cut off or start repeating itself. Then there’s the claims that this can run in real time on a CPU. That comes from a separate C++ port. It requires libtorch. It builds with metal turned off. It has zero stars. So, I don’t know what that really means. And

06:46 it doesn’t publish a single performance number. So, I wouldn’t treat real time on CPU as anything proven yet. You can be the judge of that, right? Run some tests. But it’s going to take time to figure out. And the other thing that’s gone is the whole idea of Kitty being tiny. Installing it pulls in torch transformers and diffusers. Then you need around 950 megabytes of model weights, then the chatterbox vocoder, then whisper. This version is heavier. Not only is it heavier, it’s changing very quickly. There have already been four PyPI releases in four days. There

07:18 are 120 open issues, give or take. So, this is still very early, which brings us back the full circle. Should you even try this? If you’re still building around an MVP or wanting to play around with voice, then yes, this is cool, of course. If you’re building something real, yeah, maybe not. You may just be better off paying 11 Labs or using another alternative. We’ve covered some of the best on this channel, so you could look towards Voxy M 2, for example. There are a lot of other ones out there, but we will see how this grows with time. Kitty TTS did not just

07:49 get bigger. The whole meaning of this project just completely changed. The old Kitty was tiny. That’s why it was called Kitty. Simple and Apache license. Kitty TTS 2 gives you voice cloning and much more capable speech generation, but you pay for that with size, speech dependencies, and a much more complicated license. I’m Josh from Better Stack. If you enjoy coding tips and tricks like this, be sure to subscribe. We’ll see you in another video. >> [music]