Ai2's open-source PyTorch library of building blocks and training scripts for OLMo language models, including MoE training.
Updated EU status checked Last shown
Sources: the vendor's own pages (linked above) and creator videos at the linked timestamps. EU status is information, not legal advice; see how we check.
Olmo-core is the training framework behind Ai2's fully open Olmo models. It is a PyTorch library providing building blocks for model architectures, distributed training, checkpointing and configuration, along with official training scripts for released models such as OLMo 2 32B and OLMo 3 7B and 32B. Scripts are launched with torchrun or Olmo-core's Beaker launch CLI, and most configuration options can be overridden from the command line.
The library supports several attention backends (flash-attn, ring-flash-attn, TransformerEngine), fused-linear loss via Liger-Kernel, float8 training via torchao, and dropless mixture-of-experts models via grouped_gemm. A YouTube creator describes a version ("Olmo Core 3") as an open-sourced MoE training system that keeps each expert on its own GPU and routes work to it, reporting roughly 2.7x faster training than the previous system; this claim comes from a video summary and is not verified in the provided pages. Beta autoregressive generation and a chat demo are included, and checkpoints can also be run via Hugging Face Transformers or vLLM.
It is aimed at researchers and teams who want to reproduce or modify Olmo training runs or build their own LMs. Installation is via PyPI (ai2-olmo-core) or from source, and published Docker images carry the dependencies but not the package itself. The images are tested on Ai2's H100 clusters and may not work on other hardware or driver setups.
Watch the moment: the exact second top creators show it. Videos load only when you press play, via youtube-nocookie.com.
An open-sourced training system for mixture-of-experts models that keeps each expert on its own GPU and routes work to it. It makes training about 2.7 times faster than the previous system.
Closest neighbours by meaning, not just by category.
Model · AMD
AMD's fully open 16B-parameter Mixture-of-Experts language model with 2.8B active parameters, trained on Instinct GPUs.
Model
Open model family (9B dense, 35B MoE, 397B MoE) trained with a self-improvement loop for reasoning, coding and agentic tasks.
Model
35B Mixture-of-Experts open-weights model from InternScience for long-horizon agentic search, engineering and tool use.
Library
Open-source streaming MoE inference framework for running large sparse models on laptops, phones and single-GPU PCs.
Model · Z.ai
Z.ai's open-weights 753B MoE language model, a post-trained GLM-5.2 aimed at coding, agentic tasks and cybersecurity.
Model · Accio Lab
Open-weights 35B MoE (3B active) agentic model from Accio Lab, post-trained from Qwen3.6-35B-A3B.