MAI-Base-2: Microsoft's 2.8T Model is a BREAKTHROUGH

Author: Prompt Engineer 48 · Watch on YouTube

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: MAI-Base-2: Microsoft’s 2.8T Model is a BREAKTHROUGH · Channel: Prompt Engineer 48 · 05:14 · Watch on YouTube

Topics: AI infrastructure and hardware, AI models, Multimodal models

Summary

Microsoft’s MAI-Base-2 is a 2.8-trillion-parameter mixture-of-experts model trained in a 38-day run on 16,000 GB200 GPUs using 50 trillion tokens. The video says it scored better than the open-weight models Microsoft tested on several held-out benchmarks and was four to twelve times more FLOPs-efficient than DeepSeek V4 ProBase, while noting that most competitors were post-training checkpoints. It explains the scaling methods, infrastructure and model co-design, optimization choices, and expanded text-and-vision training data behind the run. The weights are not publicly downloadable; Microsoft plans to license them to trusted partners through Foundry.

Key points

MAI-Base-2 has 2.8 trillion total parameters, with 80 billion active, and was trained on 50 trillion tokens.
The training run used 16,000 GB200 GPUs for 38 days, with 21% average model FLOPs utilization and 89% goodput.
Microsoft reports the lowest bits-per-byte scores among the models shown on held-out code, Q&A, math, and STEM.
The benchmark comparison has a caveat: most competing models are post-training checkpoints, which can shift scores.
Microsoft says its training recipe was four to twelve times more FLOPs-efficient than DeepSeek V4 ProBase and six to nearly thirteen times better than MAI-Base-1.
Source scaling makes small-model experiments better predict large-scale training by matching how often data is repeated.
The video describes jointly sweeping learning rate and weight decay, and using a multi-round dropless mixture of experts to improve expert-layer utilization.
The training process moved from AdamW to MuON and used localized gradient clipping to address outliers.
The data mix grew about 36% in unique tokens and included 4.5 trillion unique vision tokens, or 18% of the mix.
The weights are not available for download; Microsoft plans to license them to trusted partners through Foundry.

Chapters

Start	Chapter	Summary
00:00	Introducing MAI-Base-2	The video introduces Microsoft’s training run and outlines the model’s scale, hardware, and initial benchmark results.
01:04	Benchmark results and scaling forecasts	It presents held-out benchmark comparisons, notes a caveat about post-training, and explains source scaling through a small-model experiment.
02:08	Testing source scaling	A laptop experiment contrasts inaccurate forecasts without source scaling with a much closer forecast when data repetition is matched.
03:11	Model and infrastructure co-design	The video covers coupled hyperparameter tuning, dropless expert layers, MuON, and methods for reducing training bottlenecks.
04:12	Data, access, and takeaways	It describes the expanded text and vision data mix, explains that the weights will be licensed to trusted partners, and closes with Microsoft’s stated principles.

Quotes

one single training run — 00:00

Lower is better. — 01:04

error under half a percent. — 02:39

clip only where the outlier actually is. — 03:42

learn from first principles, simple methods, scale best, rigor over shortcuts. — 04:43

Concepts and entities

MAI-Base-2, Microsoft, Mixture of Experts, source scaling, DeepSeek V4 ProBase, Qwen, MuON, Foundry, GB200, AdamW

Full transcript (original language)

00:00 16,000 of the most powerful GPUs on the planet, 38 days, one single training run. And at the end of it, Microsoft says it has a base model that beats every open weight model it tested. No distillation from other models, no borrowed outputs, every training token processed in-house. This is MAI base 2, Microsoft’s frontier pre-training run, and the recipe behind it. Quick context. In June, Microsoft released MAI thinking one, built on top of MAI base one, a strong model, but

00:33 small next to the frontier. So, the plan was simple, scale up. MAI base 2 is a mixture of experts model, 2.8 trillion total parameters with 80 billion active. It was trained on a 50 trillion token mixture on 16,000 GB200s through August and September. Average model flops utilization was 21%, and good put, the share of time the run was actually making progress, was 89%. Now, the headline chart. This is bits per byte on held out code, Q&A, math,

01:04 and STEM. Lower is better. On every panel, the orange bar MAI base 2 comes in lowest below DeepSeek V4 ProBase, Chinchilla, Quen, and Mimo. And the code test is clever. It’s Microsoft’s own private code base, something no outside model could have trained on. One honest caveat, and Microsoft states it, most rivals here are post-train checkpoints, and post-training can shift these scores. But here’s the number that matters, four to 12 times more flops efficient than DeepSeek V4 ProBase. Meaning, to match DeepSeek’s quality,

01:36 Microsoft’s recipe needs a fraction of the compute, and they spend that saving on a bigger model. And compared with their own MAI base one, the gain is six to nearly 13 times in about four months. So, how did they do it? Three things, scaling science, model and infrastructure co-design, and scaling data. First, the scaling letter, a family of smaller models where the number of layers alone decides the whole architecture. Train every run with the same tokens per parameter, fit a curve, and predict the giant before you ever build it. The twist is something they

02:08 call source scaling. Small runs train on a shrunken data set repeated as many times as the big run repeats its data. Without it, small experiments reward choices that overfit at scale. With it, the forecast landed right on the final checkpoint. So, I tried the method myself on a laptop, one RTX 4060 with 8 gigs of memory. Four tiny runs from two layers up to five, same tokens per parameter byte level, so the loss is literally bits per byte. I fit the line on those four and

02:39 then forecast a six-layer model before training it. First try with no source scaling, forecast 1.33 bits per byte, actual 1.69, off by 21%. My six-layer model saw the tiny data set 20 times and overfit. Then I added source scaling, every run now repeats its own slice of data 22 times. New forecast 1.74, actual 1.73, error under half a percent. With only five runs, part of that is luck, but the failure and the fix are exactly what

03:11 Microsoft describes. Next, hyperparameters, learning rate and weight decay are coupled, so they swept them together along a diagonal band. And the law they found is beautifully simple. Learning rate times weight decay scales as one over the number of steps. Then co-design, experts in every layer always look better on paper, but ran too slowly in practice. So, they built a multi-round dropless mixture of experts, overflow tokens simply wait for an extra round, memory stays fixed, and nothing gets dropped. That lifted expert layer utilization

03:42 from 24% to 35%, enough to put experts in every single layer. They also moved from AdamW to MuON. After heavy optimization, MuON costs just 2.1% of each training step. Their theoretical floor was 55 milliseconds, they measured 165, and they named the culprits stragglers, dispatch overheads, load imbalance, and activation gradient clipping. Instead of shrinking the whole batch, clip only where the outlier actually is. Look at the gradient norm,

04:12 base one, noisy, base two, a much tighter band. And unlike most tricks they tested, its benefit grew with scale around 28% at the largest run. It even caught a data bug, a license header cut off mid-sentence, traced back to a broken preprocessing pipeline. Finally, data. The corpus grew about 36% in unique tokens with a March knowledge cut off. And this time vision is native and encoder-free design with 4.5 trillion unique vision tokens, 18% of

04:43 the mix. Image captions, charts, visual grounding, spatial reasoning, medical images, computer use, even reading sheet music. So, can you download it? No. Microsoft plans to license the weights to trusted partners through Foundry. It’s still in post-training and MAI base tree is already underway. Three principles, Microsoft says, learn from first principles, simple methods, scale best, rigor over shortcuts. 16,000 GPUs or one laptop, the latter works the same way, and that’s the real

05:14 lesson.