SageAttention

This repository is a fork of thu-ml's SageAttention that builds and distributes prebuilt Windows wheels. SageAttention is a quantized attention implementation for NVIDIA GPUs that replaces standard attention in PyTorch models to speed up inference. The fork's aim is easy installation without needing Visual Studio or the CUDA toolkit.

Wheels are published on the GitHub releases page and cover GTX 16xx, RTX 20xx/30xx/40xx/50xx, A100, H100 and AGX Orin (sm75/80/86/87/89/90/120). Recent wheels use the Python stable ABI (Python >= 3.10) and the libtorch stable ABI (PyTorch >= 2.10), so one wheel covers many versions. The user must pick a wheel matching their PyTorch version and CUDA major version (12 vs 13), and install triton-windows first. B200 and DGX Spark kernels are not bundled and require building from source. V100 is not supported.

In ComfyUI it is enabled with --use-sage-attention, though the README notes ComfyUI users can now use the SageAttention in comfy-kitchen via --use-ck-attention. Some models such as Wan and Qwen-Image may produce black or noisy output because of quantization overflow; the README suggests the KJNodes PatchSageAttentionKJ node with sageattn_qk_int8_pv_fp16_cuda or adjusting pv_accum_dtype. A YouTube creator reported about 20-30% faster MiniMax H3 generation with it.