Open-source streaming MoE inference framework for running large sparse models on laptops, phones and single-GPU PCs.
Updated EU status checked Last shown
Sources: the vendor's own pages (linked above) and creator videos at the linked timestamps. EU status is information, not legal advice; see how we check.
Edge0 is an open-source inference framework and model family for running large sparse mixture-of-experts (MoE) models on consumer hardware. It streams only the experts needed for each step from SSD into memory, using a trained router that predicts expert loads a step ahead. The project reports running a 35B MoE in roughly 2.9 GB and an 8B variant in about 1 GB, with a Recover LoRA to regain quality lost to int4 quantization.\n\nThe latest release (2026-11-13) adds edge0-145b and edge0-120b, latent-MoE models derived from Qwen3.8-Flash-Next and trained natively at 4 bits, distributed in Q3_K and NVFP4 checkpoints on Hugging Face. They run on edge0-strata (a fork of Strata) on Linux or Windows with an NVIDIA GPU (CUDA 13), or on Apple Silicon Macs via Metal. The README reports 121.2 tok/s agent decode for 145B Q3_K on an RTX 5090 D 24 GB. The earlier 35B and 8B previews run on edge0-infer, a GGML-based C++ engine that covers macOS, iOS, Android, Linux and Windows.\n\nIt is used through a CLI (edge0-infer serve and chat), an OpenAI-compatible local server, or the Edge0 Chat app examples. A technical report is on arXiv. Support varies by model and engine: the 145B/120B models are not supported on iOS or Android, and native-engine support for the 8B checkpoint is still planned. Benchmark figures come from the project's own README.
Watch the moment: the exact second top creators show it. Videos load only when you press play, via youtube-nocookie.com.
Framework that streams only needed experts of MoE models into memory, running Qwen3.5-35B in about 2.9 GB and an 8B variant in 1 GB, with a Recover LoRA to regain int4 quality loss. Runs on Mac.
Closest neighbours by meaning, not just by category.
Library · DeepSeek
Open-source inference framework from DeepSeek for serving DeepSeek models on NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs.
Model · AMD
AMD's fully open 16B-parameter Mixture-of-Experts language model with 2.8B active parameters, trained on Instinct GPUs.
Model · MiniMax
Open-weights native multimodal MoE model from MiniMax, ~428B parameters (~23B active) with 1M-token context.
Model · inclusionAI
Open-weights 560B hybrid-linear MoE reasoning model (25B active, 1M context) from inclusionAI, MIT licensed.
API · Inference.net
OpenAI-compatible inference API and gateway for open-weight and closed models, with provider failover and pass-through pricing.
Model
Open model family (9B dense, 35B MoE, 397B MoE) trained with a self-improvement loop for reasoning, coding and agentic tasks.
