YuE2: Open-Source Suno Rival With Agentic Editing

Author: Prompt Engineer 48 · Watch on YouTube

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: YuE2: Open-Source Suno Rival With Agentic Editing · Channel: Prompt Engineer 48 · 12:05 · Watch on YouTube

Topics: AI content generation, AI governance and regulation, Local and open-source models

Summary

The video tests YuE2, an open-source music-generation model that turns lyrics and a style prompt into a stereo song through an intermediate, editable score. The presenter runs it on a rented 48 GB GPU, generates a song from original lyrics, and demonstrates how changing chords in the score produces a new recording. The video also reviews the model’s licensing, benchmark results, hardware requirements, and cover workflow, while noting which features were not personally tested. YuE2 appears competitive on the cited benchmark, but the presenter cautions that the results do not establish that it beats closed systems.

Key points

YuE2 generates 48 kHz stereo audio from lyrics and a style prompt, using a score as an intermediate step.
Its pipeline includes score planning, semantic tokens, acoustic latents, and audio decoding; the score can be viewed and edited.
The code and agent skill use Apache 2.0, while the model weights are CC BY-NC 4.0 with additional creator permissions; commercial use of the weights requires contacting the authors.
On the authors’ SongBench results, YuE2 scores in a similar range to closed systems, but the presenter stresses that the benchmark does not prove it has beaten them.
The quick start calls for Linux, Python 3.12, and an Nvidia GPU with BF16 support and 24 GB of VRAM.
The presenter used a rented A40 with 48 GB of VRAM to generate a 76-second song from original lyrics in about a minute.
Changing chord symbols in the score and rendering again creates a new recording; it does not modify the original audio waveform.
The cover workflow uses Sheet Sage 2 to transcribe a recording into a melody score, but the presenter did not test that transcription step or the music agent.
The video mentions a hosted demo, ComfyUI nodes, a serving toolkit, and an agent skill as other ways to use the project.

Chapters

Start	Chapter	Summary
00:00	YuE2 overview and its score-based pipeline	The presenter introduces YuE2, its repository and licensing, and explains how lyrics and style are converted into an editable score before audio is generated.
03:06	License, benchmark, and hardware requirements	The video covers usage terms and benchmark caveats, then explains why the presenter rents a GPU instead of running the model locally.
05:13	Setup and generating songs	The presenter installs YuE2 on a rented A40, runs the example, and generates a song from original lyrics and a style prompt.
08:25	Editing chords and rendering again	The presenter compares score files with different chords and explains that editing produces a new full recording rather than altering the original waveform.
09:59	Covers, demos, and other ways to use YuE2	The video describes the cover workflow and available hosted tools, while clarifying that the presenter did not test the transcription step or music agent.

Quotes

it first writes a melody and chord plan, and only then turns that plan into a complete song. — 01:33

it is in the same range as the closed systems on this test, not that it has beaten them. — 04:09

that is the first thing I check when lyrics come out of one of these models. — 07:22

think of it as changing the sheet music and asking the band to play it again. — 09:59

I did not test the agent, I only ran the editing path by hand. — 11:33

Concepts and entities

YuE2, YuE2-3B, MAB, SongBench, Suno V5, RunPod, A40, Sheet Sage 2, YuE2 music agent skill, ABC notation, CC BY-NC 4.0

Full transcript (original language)

00:00 All right. In this video, we will see Ue2, an open music generation model where you give it lyrics and a style line and a full song with vocals and instruments comes back at 48 kHz stereo. You can see the lyrics I typed here, four lines of verse and two lines of chorus, and this is the song it gave me, 76 seconds long.

00:31 Now, mind you that my laptop cannot run this model. It has an 8 GB GPU and the model wants 24, so everything you hear was generated on a rented GPU that cost me well under a dollar. Let’s go ahead and test this out. So, this is the repo multimodal art projection/UEE. If you look here on the right, it has about 11.7 thousand stars, 1.3 thousand forks, and 13 contributors. The description says Ue2, frontier music generation with symbolic planning, zero-shot covers, and agentic music editing.

01:02 Now, the name might look familiar because the original Ue lived in this same repo and the authors have moved it to a branch called Ue version 1. So, the main branch is now all Ue2. The code is under Apache 2.0. If we go to releases, you can see the latest one is Ue2 version 0.1.6 from early September, and there is a second release for the agent skilled Ue2 music 1.2.0 from late September. The last push to the repo was on the 6th of October, so this is moving fast. Now, what makes this different from just typing a prompt

01:33 and getting audio? Here in the readme, it says that you give it lyrics and a style prompt, and it first writes a melody and chord plan, and only then turns that plan into a complete song. So, the plan is the middle step, and it is written in ABC notation, which is plain text sheet music. You can see on the architecture picture that style and lyrics become a score, then semantic music tokens, then acoustic latents, and then a decoder turns those into stereo audio. Under the hood, it is one backbone. They call it an ARN ARM mixture of transformers. The

02:04 autoregressive part predicts the score and the semantic tokens. And then flow matching generates the acoustic latents and AVAE decodes those into audio. The Python side exposes exactly those stages: plan, generate semantics, synthesize, and decode one after the other. And the reason this matters is the setting called COT. If you set it to full, which is the default, it generates the plan and you can read it. If you set it to melody, it plans only the melody and leaves the accompaniment free. And that is what they recommend for covers.

02:34 And off means it goes straight from lyrics and style to audio. So, this is the thing we are going to poke at in this video because a score you can read is a score you can change. Before we run anything, the license. Because this is the part people get wrong. The code and the agent skill are Apache 2.0. The model weights, which you can see here on the Hugging Face page for U E23B, are CC BY NC 4.0 with an additional creator permission. And the readme table says it like this. Personal users, content creators, and musicians can use U E2 to

03:06 monetize what they generate with no fees or royalties to the authors. Academic research and education is free for non-commercial use. And if you’re a company that wants to use the weights commercially, you have to contact them for a commercial license. So, the songs are yours to use, but you cannot just build a paid product on top of the weights. That is my reading of the table on screen. So, go and read the full terms file before you do anything serious. Now, the claim that got this repo trending on their benchmark while song bench, which is 192 prompts with

03:38 automatic scoring from the 12th of September UE2, with best of eight samples score 6.96 and Miracle 9 score 6.94 and Suno V5 score 6.87. The normal UE2 setting, which picks from two candidates, scores 6.73, which puts it just under Suno V5 and above Suno V5.5 at 6.71 and Suno V6 at 6.56. And the old UE from the first version scores 4.9 on the same benchmark. So, the jump from version one is big. Mind

04:09 you, this is the author’s own benchmark, and they say it themselves right under the table that rankings change depending on the metric and that the small gap between the top means does not establish statistical significance. So, what I take from it is that it is in the same range as the closed systems on this test, not that it has beaten them. The only way to know for your own taste is to listen, and they actually run a public listening arena for that with a link on the repo. So, the quick start says Linux Python 3.12 and an Nvidia GPU with BF16 support and 24 GB of VRAM

04:43 because it produces 48 kHz stereo without quantization. Now, if you look at my machine here, Nvidia SMI says RTX 46 laptop with 8,188 MB, that is 8 GB. So, it is a third of what the model asks for. On top of that, I’m on Windows, and the Windows subsystem for Linux has no distribution installed on this laptop, and my D drive has only 2 and 1/2 GB free. So, the weights would not even fit. So, I did not try to squeeze it in. I rented an A40 on RunPod, which has 48

05:13 GB of VRAM at 59 cents an hour. Now, for the setup, this is just a quick start from the readme. The pod comes with Python 3.112 already, so I clone the repo, make a virtual environment, upgrade pip, and run pip install. You can see in the log that the install took 1 minute and 30 seconds. Then, the command is python examples/generate.py with an output folder. On the first run, it fetches 11 files of model weights from Hugging Face, and the whole first run download included took 1 minute and

05:44 1 second. And the script prints the path to audio.flac and says truncated is false for both the score and the semantic tokens, which means the song was not cut off. This is the first song from the example request in the repo, warm piano pop at 88 beats per minute, and it is 59.9 seconds long, 48,000 samples per second, two channels. And the output folder keeps everything, not just the audio. >> [music]

06:20 [music] >> The old [singing] face along the lane. Footsteps keep the time of [music] rain. Fold the night and leave it here. [singing and music] Morning has a sky [singing] to [music] clear. Let the day come into view. >> You can see the score.abc, the plan, the

06:51 semantic tokens, the latent file, and the request and result JSON. So, if you ever want to know what a song was made from, it is all sitting there next to it. Now, I wanted to try my own lyrics, so I wrote a verse and a chorus about renting a GPU, and the style line says upbeat indie rock, energetic male vocal, electric guitar, bass, and live drums, 120 beats per minute with a fixed seed. This one took 60 seconds to generate and came out 76 seconds long. So, on this A40, it was about real time, and listen to how the words land against the melody

07:22 on the chorus, because that is the first thing I check when lyrics come out of one of these models. >> [music] [music] >> One card [music] in the cloud, rented by the hour. >> [music] >> My laptop is too small, so I borrowed

07:55 the power. Type in the lyrics [music] wait a minute or two, a whole song comes back vocals and all. Sing it, sing it. From a prompt to a song. Sing it, sing it. All night long. >> Now the editing part, which is the reason I’m excited about this. The repo

08:25 has two score files, score.abc and scorejazz.abc. And the only difference is the chord symbols. If you look at the diff here, C becomes C major 7, G becomes G7, A minor becomes A minor 7, and F becomes F major 7. The notes, timing, meter, and tempo are the same. There is a helper script that checks this, and when I ran compare on the two files, it says match true with an empty list of differences. And it carries a note that it only claims the melody and meter are the same. It

08:56 makes no claim about the audio. Then I generated both with the same settings, and the first one took 31 seconds and gave 22 seconds of audio. And the jazz one took 30 seconds and gave 24. >> Neon fades [music and singing] along the lane. Footsteps keep the time of rain. Fold [singing] the night and leave it here. [music] Morning has a sky to clear. Let the [singing] day come into view. Every [music] road begins with you. Hold a little room [singing] for light.

09:27 We will sing beyond the night. Neon fades [music] along the lane. Footsteps keep the time of rain. [music] Fold the night and [singing] leave it here. Morning has a sky to clear. Let the day come into [music] view. Every road begins with you. Hold a little [music] room for light. We will sing beyond the night. [music] >> Now, this is what the docs say honestly that editing makes a new full recording.

09:59 So, the singing and the timbre can change even where the score did not. It does not touch up the old waveform. So, think of it as changing the sheet music and asking the band to play it again. Now, covers. The way this works is a second model. Sheet Sage 2 transcribes a recording into a melody score and then Yui 2 takes that score with new lyrics or a new style. On their numbers across 948 works, the full score version reaches .647 on a retrieval metric called CLE dubious MAP versus .006 with

10:29 no score at all. I did not run that transcription step because Sheet Sage 2 needs its own Python 3.10 or 3.11 environment with FFmpeg 6.1. So, I’m not going to claim I tested a real cover. What I did run is the included original melody fed in as a score in melody mode and it took 37 seconds for 37.5 seconds of audio. If you want to hear actual zero-shot covers, the author’s demo page has them and I have put the link in the description. >> Neon fades along the lane. [music]

11:01 Footsteps keep the time of rain. Hold the night and leave it here. Morning has a sky to clear. Let the day [music] come into view. Every road begins with you. Hold [singing] a little room for light. We will sing beyond the night. >> [music] >> And if you You not want to touch Linux at all, the new section lists a free

11:33 hosted demo from noise native comfy UI nodes with an official text-to-music workflow and UE2 Turbo a serving toolkit that handles concurrent requests. There’s also the UE2 music agent skill where an agent does the score editing for you and the repo recommends GPT-6 Astra as that agent. I did not test the agent, I only ran the editing path by hand. So, in this video we have seen how to run UE2 on a rented 48 GB GPU, make a song from my own lyrics in about a

12:05 minute and change the chords of a score and render it again. If you’re interested, you can go ahead and check this out and try this out on your own system as well. If you have a 24 GB card and let me know how it goes. The model is UE2-3B from MAB and all the links are in the description. I’ll see you in another one. Check out the next video.