Ai2 Releases Olmo-Core 3, Open Training Stack for Trillion-Parameter MoEs

The Allen Institute for AI released Olmo-core 3 on October 1, 2026, an upgrade to its large language model development framework built around a redesigned open mixture-of-experts training system that Ai2 said it has benchmarked at more than one trillion total parameters. Ai2 said Olmo-core 3 is designed to scale MoE training into the trillion-parameter range while preserving computational efficiency, and described it as one of the core systems behind the next generation of Olmo.

The release is accompanied by a technical report, an interactive demo, and open code. MoE models can contain many more parameters without requiring every input to use all of them, but the full model still has to be stored across GPU memory and updated during training, and routing inputs to the right experts across a cluster creates its own communication and coordination costs.

Ai2 said those costs can erode much of the computational advantage as MoEs grow, and that Olmo-core 3 is built to close that gap. In one Ai2 benchmark, the expert pool grew from 8 to 128 while still selecting four experts per…