ERNIE 5.5 Flash

ERNIE 5.5 Flash Base is a pretrained-only language model from Baidu, released on 5 November 2026. It is a sparse Mixture-of-Experts model with 280B total parameters and 14B activated per token, 43 Transformer layers, hidden size 4096 and 64 attention heads. It was pre-trained from scratch on 33.5T tokens with a training context of up to 128K, and supports a 1M-token context for fine-tuning and inference.

The weights are the end-of-pretraining checkpoint, with no mid- or post-training. Baidu positions it as a starting point for domain adaptation, instruction tuning and reinforcement-learning research, and as a controlled baseline for comparing training methods. The architecture combines Multi-head Latent Attention in one layer out of seven with Heavily Compressed Attention (128x compression) in the rest, DSA sparse attention in the full-context layers, low-rank gated attention and Virtual Head Attention, to reduce KV-cache and long-context cost. The pre-training corpus emphasises repository-scale code and long-context understanding.

A YouTube review reports FP8 (~288 GB) and BF16 (~573 GB) weight releases and describes it as strong among open models of similar size. Because it is a base model, it is not instruction-tuned and is aimed at developers and researchers with significant GPU resources. The provided pages did not state the license, download location or EU-specific terms.