AMD's fully open 16B-parameter Mixture-of-Experts language model with 2.8B active parameters, trained on Instinct GPUs.
Updated EU status checked Last shown
Sources: the vendor's own pages (linked above) and creator videos at the linked timestamps. EU status is information, not legal advice; see how we check.
Instella-MoE (Instella-MoE-16B-A3B) is a decoder-only Mixture-of-Experts language model from AMD with 16B total and 2.8B active parameters per token. It has 27 decoder layers, a hidden size of 2048, and a shared-plus-routed expert design (2 shared experts, 6 routed experts chosen from 64 per token). It uses Gated Multi-head Latent Attention (Gated MLA) and FarSkip-Collective connectivity, which AMD says sped up pre-training by 12.7% by overlapping expert-parallel communication with computation.
The model was trained from scratch on AMD Instinct MI300X and MI325X GPUs with the ROCm software stack, using the open-source Primus training framework and Miles RL framework. AMD releases checkpoints for every stage: pre-training, mid-training, long-context base, SFT, DPO, and an RL-refined Think checkpoint. It also publishes training configurations, data mixtures and code. AMD reports competitive benchmark results against dense and MoE models of similar size, though these are vendor-reported claims.
It is aimed at researchers and developers who want a reproducible, fully open model and a reference for training MoE models on AMD hardware. The source material does not give a license, context length or exact release date.
Watch the moment: the exact second top creators show it. Videos load only when you press play, via youtube-nocookie.com.
Open-source 16B MoE (2.8B active) model trained from scratch on AMD Instinct GPUs with the ROCm stack. It reportedly beats similar-sized models such as Gemma 4 E4B, and AMD published checkpoints, recipes and code.
Closest neighbours by meaning, not just by category.
Model · Rednote
Open MoE language model from Rednote (Xiaohongshu) with 208B total and 16B active parameters.
Model · MiniMax
Open-weights native multimodal MoE model from MiniMax, ~428B parameters (~23B active) with 1M-token context.
Model · Z.ai
Z.ai's open-weights 753B MoE language model, a post-trained GLM-5.2 aimed at coding, agentic tasks and cybersecurity.
Model · inclusionAI
Open-weights 560B hybrid-linear MoE reasoning model (25B active, 1M context) from inclusionAI, MIT licensed.
Model · Meituan
Meituan's open-weights 1.6T-parameter multimodal MoE model (48B active) for agentic coding and visual tasks.
Model · Alibaba
Alibaba's open-weights 2.4T-parameter MoE language model (95B active) from the Qwen3.8 family, with a hosted Max version.
