AI news ·
Microsoft completes pre-training of MAI-Base-2, a 2.8 trillion parameter foundation model
Microsoft trained MAI-Base-2, an 80B-active-parameter model, on 16,000 GPUs over 38 days and claims 4-12x better pre-training efficiency than DeepSeek-V4-Pro-Base.
Microsoft AI has completed pre-training for MAI-Base-2, a new mixture-of-experts foundation model with 80 billion active parameters and 2.8 trillion total parameters. The model outperforms every tested open-weight model on held-out evaluations while achieving 4-12x greater pre-training FLOPs efficiency than DeepSeek-V4-Pro-Base, the strongest open base model available.
MAI-Base-2 trained on 16,000 GB200 GPUs over 38 days in August and September 2026. The run averaged 21% model FLOPs utilization and 89% goodput. Microsoft said owning the full model pipeline end-to-end - from data provenance through infrastructure - allows it to offer enterprise customers "a trusted source of world-class intelligence" through its Foundry platform.
Efficiency gains through scaling science
The team used a technique called source scaling to make small-scale experiments reliably predict large-scale behavior. By artificially restricting unique training samples on lower rungs of a scaling ladder, each dataset repeats the same number of times regardless of training horizon. This keeps the ratio of unique data to parameters fixed and prevents smaller experiments from favoring choices that overfit at scale.
Microsoft estimated its recipe is 4.2 to 12.3 times more FLOPs-efficient than DeepSeek-V4-Pro-Base, and 6.3 to 12.7 times more efficient than its own MAI-Base-1 released in June. The team attributed these gains to data quality and modeling choices rather than simply scaling compute.
New scaling laws for hyperparameters also contributed. The team jointly optimized learning rate and weight decay, finding the product of the two depends primarily on training steps while the learning rate alone also depends on tokens per active parameter. A separate scaling law for dropout emerged from the same experimental approach.
Infrastructure and model co-design
MAI-Base-2 uses mixture-of-experts in every layer, a shift from MAI-Base-1's alternating pattern. The team fixed inefficiencies in dynamic routing and per-expert overhead that previously made all-layer MoE suboptimal. A multi-round dropless MoE implementation eliminates memory fragmentation by grouping experts and processing overflow tokens in additional rounds, keeping memory usage static regardless of routing imbalance.
The model also adopts the Muon optimizer over AdamW. Muon orthogonalizes momentum matrices using Newton-Schulz iteration, which requires resharding momentum across GPUs. After optimization, Muon accounted for 2.1% of average training iteration time - 165 milliseconds - compared to a theoretical minimum of 55 milliseconds bottlenecked by communication.
Activation gradient clipping replaced standard global gradient clipping. By clipping gradients at selected points in the network, the technique prevents outlier examples from scaling down gradients across an entire batch. The benefit grew with model scale, and the team used it to catch a preprocessing bug in code data when a license header was abruptly cut off mid-sentence.
Organic data at larger scale
MAI-Base-2 follows the same data principles as its predecessor: no distillation from other models, with every training token processed in-house. The training corpus grew roughly 36% from MAI-Base-1 in unique tokens. Microsoft expanded web content pipelines with a March 2026 knowledge cutoff and used embedding-based diversity sampling to prioritize novel data.
Unlike MAI-Base-1, the new model pre-trains natively with vision using an encoder-free VLM architecture. The vision-language corpus contains 4.5 trillion unique tokens across image-text pairs and interleaved documents. With repetition, VLM data accounted for 18% of the 50-trillion-token training mixture.
Why this matters for executives and technical leaders
MAI-Base-2 signals Microsoft's bet that owning the full stack - data, model architecture, and training infrastructure - compounds into measurable efficiency advantages. The 4-12x FLOPs efficiency claim, if validated in post-training and product integration, translates directly to lower inference costs and faster iteration cycles. For enterprises evaluating AI partnerships, the emphasis on organic data with strong provenance addresses growing concerns about training-set contamination and licensing risk. Teams building their own training pipelines can study the open discussion of source scaling, dropless MoE, and activation gradient clipping as practical techniques, while those pursuing AI technology leadership should track whether the post-trained version of this model delivers on the base model's benchmarks.