RTX 5090 vs M5 Ultra: Which AI Powerhouse Wins?
Read the Portuguese translation →
Author: Hugging Face · Watch on YouTube
Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.
Original language: English
Original video: RTX 5090 vs M5 Ultra: Which AI Powerhouse Wins? · Channel: Hugging Face · 03:08 · Watch on YouTube
Topics: AI evaluation and benchmarks, Model inference and optimization, AI infrastructure and hardware, Local and open-source models
Summary
Section titled “Summary”The video compares the Apple M5 Ultra with the RTX 5090 on specifications, power consumption, and AI model performance. In the cited Qwen 3.8 27B 4-bit quant benchmark, the RTX 5090 is faster for prompt processing and token generation. The M5 Ultra, however, supports much more memory, allowing it to run larger models. The video suggests choosing based on whether performance or memory capacity is the priority, while noting that the estimates may change as AI engines evolve.
Key points
Section titled “Key points”- The M5 Ultra has 1.2 TB/s of memory bandwidth, compared with 1.8 TB/s for the RTX 5090.
- The M5 Ultra can currently be ordered with 256 GB of RAM, with 512 GB described as coming soon; the RTX 5090 has 32 GB of VRAM.
- The stated compute performance is approximately 110 TFLOPS for the M5 Ultra and 419 TFLOPS for the RTX 5090.
- At sustained peak load, the M5 Ultra consumes about 214 W, while the RTX 5090 draws 300–575 W.
- For the Qwen 3.8 27B 4-bit quant model with a 6K prompt, prompt processing is about 3,000 tokens/s on the 5090 and 1.7K tokens/s on the M5 Ultra.
- Time to first token is reported as 2 seconds for the 5090 and 4 seconds for the M5 Ultra.
- With MTP off, generation is about 75 tokens/s on the 5090 and 52 tokens/s on the M5 Ultra; with MTP on, the figures are 136 and about 100 tokens/s, respectively.
- The video recommends the 5090 when a model and its overhead fit within 32 GB, and notes that the M5 Ultra can handle larger or unquantized models and mixture-of-experts models.
Chapters
Section titled “Chapters”| Start | Chapter | Summary |
|---|---|---|
| 00:00 | Specifications and memory | The video introduces the comparison and contrasts the devices’ memory bandwidth and available memory. |
| 01:03 | Power and benchmark setup | It compares sustained peak power draw and introduces the AI model used for performance figures. |
| 01:34 | AI inference performance | The video compares prompt processing, time to first token, and token generation with MTP off and on. |
| 02:36 | Choosing a device for local models | It explains how memory capacity and model size can guide the choice between the RTX 5090 and M5 Ultra. |
Quotes
Section titled “Quotes”The M5 Ultra has a memory bandwidth of 1.2 terabytes a second. — 00:00
The time to first token is 2 seconds and 4 seconds, respectively. — 01:34
Remember, these are just current estimates based on the fast-moving changes that are happening with the different types of engines that can run AI models. — 02:05
If the model plus any overhead that it needs can fit within 32 GB, then the 5090 will get you great performance. — 02:36
Concepts and entities
Section titled “Concepts and entities”M5 Ultra, RTX 5090, Mac Studio, Qwen3.8 27B, MTP, inference, Mixture of Experts
Full transcript (original language)
Section titled “Full transcript (original language)”00:00 Apple just started shipping the Mac Studio with the M5 Ultra. In this video, we’re going to have a look at the important specs for the M5 Ultra, compare them to the RTX 5090, and then we’re going to look at performance numbers that people are getting for both of these in an apples-to-apples comparison. Let’s have a quick look at the specifications of these devices. The M5 Ultra has a memory bandwidth of 1.2 terabytes a second. The RTX 5090, 1.8 terabytes a second.
00:32 The M5 Ultra you can currently order with 256 gigs of RAM, with the ability to order 512 gigs coming soon. The RTX 5090 has 32 gigs of VRAM. In terms of TFLOPS or compute performance, the M5 Ultra gets approximately 110 TFLOPS, whereas the 5090 can reach 419 TFLOPS. Power consumption is also substantially different between these two devices. The M5 Ultra
01:03 consumes about 214 watts at sustained peak load, whereas the 5090 will reach anywhere from 300 to 575 watts of power draw during sustained peak load. Now, let’s have a look at how fast both of these devices will run an AI model. We’re going to be looking at numbers for Qwen 3.8 27B 4-bit quant 6K prompt. The 5090 does
01:34 prompt processing at about 3,000 tokens a second, whereas the M5 Ultra processes at about 1.7K tokens a second. The time to first token is 2 seconds and 4 seconds, respectively. Token generation with MTP off on the 5090 is about 75 tokens a second, and on the M5 Ultra, about 52 tokens a second. With MTP on, the 5090 gets a blazing 136
02:05 tokens per second and the M5 Ultra, about 100 tokens per second. Remember, these are just current estimates based on the fast-moving changes that are happening with the different types of engines that can run AI models. As you already know, the RTX 5090 comes with 32 GB of RAM. The M5 Ultra, however, can be ordered with up to 256 gigs of RAM current. That means you can
02:36 run much larger models on the M5 Ultra. Remember these things if you want to run models locally. The inference speed or the tokens per second is typically the bandwidth divided by the model size. If the model plus any overhead that it needs can fit within 32 GB, then the 5090 will get you great performance. If you have a bigger models, unquantized models, or you’re running a mixture of experts model, the M5 Ultra
03:08 can be a great choice for that. Leave your comments below and let me know what other videos you’d like to see on running AI models locally.