Qwen 2.5 0.5B

Qwen2.5-0.5B is the smallest model in Alibaba's Qwen2.5 family of language models, which ranges from 0.5B to 72B parameters. The listed repository contains the pretrained base model, a causal language model with about 0.49B parameters (0.36B non-embedding), 24 layers, grouped-query attention (14 query and 2 key/value heads), RoPE, SwiGLU, RMSNorm and tied embeddings.

The Qwen2.5 series is described as improving on Qwen2 in knowledge, coding, mathematics, instruction following, long-text generation, structured data understanding and JSON output. The 0.5B base model has a 32,768-token context length and supports over 29 languages, including Chinese, English, French, Spanish, German and others. The model card advises against using the base model directly for conversation and recommends post-training such as SFT or RLHF, or using the instruction-tuned variant.

Weights are downloadable from Hugging Face and ModelScope and are usable with the Transformers library (a recent version is required) and local runtimes such as llama.cpp, Ollama and LM Studio. Its small size makes it suitable for experimentation, fine-tuning, edge or CPU use, and teaching; YouTube creators have used it in a free Colab demo to contrast a base model with an instruct model. Its capability is limited compared with larger Qwen models. Note that the GitHub repository now covers the newer Qwen3 series.