ACE-Step

ACE-Step is an open-source foundation model for music generation. It combines diffusion-based generation with Sana's Deep Compression AutoEncoder (DCAE) and a lightweight linear transformer, and uses MERT and m-hubert representations for semantic alignment during training. The project reports synthesis of up to 4 minutes of music in about 20 seconds on an A100 GPU, which it describes as 15× faster than LLM-based baselines.\n\nIt generates songs from text style prompts and lyrics, supports 19 languages (with uneven quality for less common ones), and can produce instrumentals. Training-free controls include retake (variations), repaint (regenerating a section) and lyric editing. LoRA fine-tunes are provided for applications such as Lyric2Vocal, Text2Sample and RapMachine. StemGen and Singing2Accompaniment are listed as coming soon.\n\nCode and weights are published on GitHub and Hugging Face under Apache-2.0, and can be run locally (the README cites a reduced VRAM requirement of about 8GB with offloading options), with a ComfyUI node and a Gradio UI available. The authors list limitations: output varies strongly with random seed and duration, weaker results on some genres, artifacts in repaint/extend transitions, coarse vocal quality and limited fine-grained control. A newer ACE-Step v1.5 was released on 2026-01-28.