Fish Audio Drama 3: Can You Run It Yourself?

Author: Prompt Engineer 48 · Watch on YouTube

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: Fish Audio Drama 3: Can You Run It Yourself? · Channel: Prompt Engineer 48 · 07:43 · Watch on YouTube

Topics: AI evaluation and benchmarks, AI governance and regulation, AI infrastructure and hardware, Local and open-source models, Multimodal models

Summary

The video tests whether Fish Audio’s 35-billion-parameter Drama 3 text-to-speech model can be run on rented hardware and clone the creator’s own voice from a 30-second sample. An 8 GB laptop cannot run the model, so the creator uses a 48 GB A40 GPU and resolves four setup problems before generating speech. The voice cloning works well for a first attempt, though performance is slower than the reported H200 benchmark and the real cost includes startup and idle time. The video also notes licensing, optimization, and testing limitations, and advises checking the license before commercial use.

Key points

Drama 3 has about 35 billion parameters in total, with roughly 3 billion active per token, and supports text directions and up to 16 speaker tags.
The creator’s 8 GB RTX 4060 could not run the available weights, so the test used a 48 GB A40 on RunPod for $0.59 per hour.
Four setup issues required fixes: a system-locked Python environment, a Hugging Face Hub version conflict, insufficient vocoder frame capacity, and a startup timeout.
Voice cloning required a reference clip of no more than 60 seconds and a word-for-word transcript; the creator used their own 30-second recording.
The generated voice sounded similar to the creator, but expressive details such as laughter and whispering were not always right on the first try.
The A40 generated about 430 seconds of audio in about 165 seconds, or roughly 0.38 times real time, slower than the reported H200 benchmark.
At the measured generation speed, one million characters would cost about $3.60 in GPU time, excluding startup and idle time.
The video cautions that license information differs across parts of the model page and recommends reading the license file before shipping a product.
The creator tested only English and did not test 16-speaker scenes or the invite-only editing feature.

Chapters

Start	Chapter	Summary
00:00	Drama 3 and the reported benchmarks	The creator introduces the voice-cloning test, describes the model’s capabilities, and reviews reported benchmark and Hugging Face page details.
02:02	Choosing a GPU and fixing the setup	After ruling out local hardware and a 24 GB GPU, the creator runs Drama 3 on an A40 and resolves four installation and startup problems.
04:05	Testing voice cloning and generation speed	Using a 30-second reference and its transcript, the creator tests plain speech and delivery tags, then compares generation speed with the benchmark.
06:12	Costs, license, and limitations	The creator estimates GPU costs, explains the license caveats and untested features, and summarizes what is needed to run the model.

Quotes

I rented one GPU for $0.59 an hour and I’m going to show you what broke before it worked. — 00:00

I set it to 2,400 and on the next start, it came up healthy. — 03:34

That is about 0.38 of real time. — 05:39

So, your real cost is higher. — 06:12

I only tested English even though the model list 83 languages — 07:12

Concepts and entities

Drama 3, Fish Audio, RunPod, A40, RTX 4060, H200, Hugging Face, Hugging Face Hub, Transformers, int4, FP8, S2.1 Pro, Qwen3

Full transcript (original language)

00:00 This voice is not me talking. It is a 35 billion parameter model called Drama 3 and it learned my voice from 30 seconds of audio. I rented one GPU for $0.59 an hour and I’m going to show you what broke before it worked. Mind you, the sample is my own voice so there is nobody being cloned without consent. Stay for the speed number at the end because it is slower than the benchmark says and I’ll explain why. So this is the Drama 3 page on Hugging Face from

00:30 Fish Audio. It is a text-to-speech model with about 35 billion parameters in total but is a mixture of experts so only about 3 billion are active for each token. You can see the speaker tags here from to speaker 15. So one piece of text can hold 16 characters. You write directions in square brackets like whispering and you can change the direction in the middle of a line. And the model card lists 83 languages. For cloning, you give it a reference clip and the exact

01:00 transcript of that clip and that’s what I did with my own 30-second sample. Now let’s check what Fish Audio says on their own blog. They say it scores 93.2% on the artificial analysis pronunciation benchmark against 88.1 for their previous model S2.1 Pro. That is an outside leaderboard but the number comes from their blog and I did not run it myself so take it as reported not verified. The technical report on archive says a single stream decodes at 271

01:32 frames per second on one H200 compared to 177 for the older S2 model even though this one is more than seven times bigger. And that is an H200 which costs several dollars an hour. Keep that in mind because I’m on a much smaller card. Back on the hub page, it has about 490 likes, 4,040 downloads last month and 27 quantized versions and no inference provider hosted right now. So you either run it yourself or you pay Fish Audio.

02:02 Their API is $50 per million characters and $10 until November 12th. Remember those two prices because I’ll compare them to my own cost. First question is whether I can run this locally. My laptop has an RTX 4060 with 8 GB of memory. The SDLang Omni Cookbook says the default weights take 37 GB and the N4 version takes 20 point. So, even the small one is more than two and a half times my memory. That is a no, so I went to RunPod. So, which card? Not a 4090

02:32 and here is the math. The server gives the engine 70% of the card and on a 24 GB 4090, that is 16.8 GB less than the 20.8 the weights need. The A40 has 48 GB and 70% of that is about 32. The four-bit build called int4 also needs compute capability 8.0 or newer and the A40 is 8.6. It cost 59 cents an hour. Now, here’s where it got honest. The install failed in under a second because

03:03 the pod’s Python is locked by the system, so I made a virtual environment. Then the server crashed with an import error because I had upgraded Hugging Face Hub to version two and the Transformers library wants version below two. I pinned it back. The third one, the server refused to start because the vocoder only had 4.4 GB which fits 1,239 frames and the default asks for 2,048. I cut the new token limit to 1,024,

03:34 which is about 47 seconds of audio per request and I split my script into short pieces myself. And the fourth one was a timeout. The weights load from a network disk about 76 seconds for the first shot of 12 and the server gives up after 600 seconds. There is an environment variable for that. I set it to 2,400 and on the next start, it came up healthy. So, budget some patience on a first start. Here’s the request. I send the text plus a reference with the path to my wave file and the

04:05 transcript of what I say in it. Two rules from the cookbook. Each reference is kept at 60 seconds. Anything longer comes back as an HTTP 400 error, and mine is 30.2 seconds. And the transcript has to match what is spoken word for word, so I pasted my own text file. First line, plain text about the model release. Fish audio just released Drama 3, and it is a 35 billion parameter text-to-speech model that you can actually run yourself. Second line, I

04:36 added a laughing tag in the middle of the sentence. I thought there was no way this would work on a single A40. And then it spoke in my own voice. Third line, I switched from calm to whispering to serious in one go. Here is the claim, but hold on. This is where it gets strange. The numbers do not lie. You can hear it is my voice, same accent, same pace, and I did not train anything, no fine-tuning. It is just that 30-second clip. And now, where it is not perfect,

05:07 the model decides how each text sounds. So, the laughing is not how I laugh, and I would still regenerate a line if the whisper came out too loud. But for a first try with no setup on my side, it is already usable for a video, and this whole voice-over is made the same way. Now, the numbers. The cookbook says int4 is about 8% slower than fp8, 1.234 seconds against 1.235, and the word error rate is 1.63% against 1.99, which they say is within

05:39 run-to-run noise. On my A40, the first line took 11.7 seconds for 9 seconds of audio because it was warming up. The next two took 2.7 and 2.6 seconds for about 6 seconds of audio each. Then I generated this whole voice-over, 46 pieces, about 7,500 characters, and it made about 430 seconds of audio in about 165 seconds, retries included. That is about 0.38 of real time. The benchmark says 0.13 on an H200, so this is the

06:12 slower number I promised you. Now, the money. At that speed, 1 million characters takes about 6 hours of GPU time, which is about $3.60 at $0.59 an hour against $50 on the API or 10 with the discount, but that only counts the time it was generating. It does not count the long first start or the hours the pod sits idle. So, your real cost is higher. Three limits. One, the license. It is free for research and non-commercial use and free for commercial use if you and your

06:42 affiliates make under $500,000 a year. Above that or if you offer it as a cloning service to other people, you need a license from Fish Audio and cloning someone’s voice without consent is not allowed. One flag, the license tag at the top of the model card says Fish Audio Research license, while the blog and the card’s own license section say community license. So, read the license file before you ship anything. Two, the server log itself says the int4

07:12 build is not fully optimized yet. Three, I only tested English even though the model list 83 languages and I did not test the 16 speaker scenes either. And the editing feature that fixes one word in a take, the blog says it is in invite-only testing and an editing API is planned, so I could not try that one. So, can you use this for your own voice-over? Yes, if you have a 48 GB card, a 30-second clean sample with its exact transcript and some patience on

07:43 the first start. It took me four fixes in an A40 at $0.59 an hour. If you are looking for cheap and reliable GPUs, I use RunPod and the link is in the description. Next, I’ll put this model against my old Queen 3 clone, same script, same voice, and we’ll see which one wins. Let me know in the comments which line you want me to test and if you want the exact commands and the four fixes, they are in the description.