MiniMax's open-source sparse top-k and dense attention GPU kernels for NVIDIA SM100 (Blackwell), with a technical report.
Updated EU status checked Last shown
Sources: the vendor's own pages (linked above) and creator videos at the linked timestamps. EU status is information, not legal advice; see how we check.
MiniMax Sparse Attention (MSA) is a sparse attention mechanism from MiniMax, released as a Python package (fmha_sm100) with GPU kernels. According to a creator summary, it uses a lightweight indexing branch that selects the top memory chunks before attention is computed, which makes long-context processing more efficient. The repository includes dense FlashAttention-style kernels and sparse top-k attention kernels.
The package has two JIT-compiled stacks. The csrc stack provides dense FMHA and a sparse_topk_select indexer, compiled from Jinja templates at runtime. The CuTe-DSL stack provides full sparse attention, covering forward passes and paged FP8 decode, with BF16, FP8, NVFP4 and FP4 support. A bridge module adapts the fmha_sm100 API to the sparse attention path for prefill. The repository also includes tests and a benchmark script, and the kernels can be loaded through Hugging Face's kernels library.
It targets inference and kernel developers working with long-context models. Requirements are an NVIDIA SM100 GPU, a CUDA Toolkit with nvcc, Python 3.10 or newer, and Linux x86_64 (aarch64 is untested). The first run JIT-compiles kernels, which can take from 30 seconds to a few minutes. The code is MIT-licensed, while bundled third-party code such as NVIDIA CUTLASS keeps its own licenses.
Watch the moment: the exact second top creators show it. Videos load only when you press play, via youtube-nocookie.com.
Attention mechanism with a lightweight indexing branch that selects top memory chunks before attention, enabling efficient long context. Code and technical report released.
Closest neighbours by meaning, not just by category.
Model · MiniMax
Open-weights native multimodal MoE model from MiniMax, ~428B parameters (~23B active) with 1M-token context.
Model · Hao AI Lab
Open-weight 4-step sparse-distilled version of MiniMax H3 for fast text-to-video-and-audio generation on Blackwell GPUs.
Model · Nvidia
NVIDIA's 5B and 14B hybrid linear-attention video diffusion transformers generating up to 720p on a single GPU.
Model · Unsloth
GGUF quantizations of MiniMax-H3 video-with-audio generation model, for stable-diffusion.cpp and Unsloth.
Model · MiniMax
Open-weights MiniMax image generation and editing model producing up to ~16 MP images in 6 sampling steps.
Model · Kijai
Community-quantized W4A8 and int8 VAE checkpoints of MiniMax H3 video model for ComfyUI, by Kijai.
