Vision Transformer foundation model for sheet music, pre-trained with masked autoencoding on 9.7M IMSLP pages.
Updated EU status checked Last shown
Sources: the vendor's own pages (linked above) and creator videos at the linked timestamps. EU status is information, not legal advice; see how we check.
MuSViT is a vision foundation model for sheet music representation, developed by researchers at the University of Alicante (GRFIA group) and presented in an ECCV 2026 paper. It uses a ViT with 12 Transformer layers, 16x16 patches projected to 768-dimensional embeddings, and 2D sinusoidal positional encodings intended to capture vertical position, which matters for reading staff lines and pitch.
The model is pre-trained with masked autoencoding (MAE) on 9.7M pages from IMSLP through a two-stage synthetic-to-real curriculum. According to the authors, it outperforms general-purpose vision encoders under linear probing across four downstream tasks, and surpasses task-specific state of the art on three of the four under fine-tuning. Their embedding-transcription analysis reports correlation above 0.6 with symbolic musical content, while general-purpose encoders produce anti-correlated embeddings.
Model weights, pre-training code and evaluation scripts are stated to be publicly released. It is an encoder backbone for researchers and developers building optical music recognition and related sheet-music tools, not an end-user application. The exact repository and license were not given in the provided material.
Watch the moment: the exact second top creators show it. Videos load only when you press play, via youtube-nocookie.com.
A vision model for reading sheet music, trained with masked reconstruction on 9.7M pages covering about 400k works. It beats other vision models at recognizing and classifying sheet music and is small enough for consumer devices; released openly.
Closest neighbours by meaning, not just by category.
Model
Open-source model that transcribes songs into separate instrument tracks and MIDI notes, including vocals.
Model · AIDC-AI
Open multimodal embedding models from AIDC-AI that map text, images, video and documents into a shared vector space.
Model · ByteDance
Research zero-shot TTS model that generates speech by modeling raw waveforms directly with flow-matching DiT.
Model · ACE Studio
Open-source local music generation model with a 4B-parameter DiT decoder, extending ACE-Step 1.5 with higher audio quality.
Model · Edge0
Open-weights 0.17B-parameter multilingual text-to-speech model with zero-shot voice cloning, runs on CPU via ONNX.
Model · Multimodal Art Projection (M-A-P)
Open-weights music generator that plans an editable ABC-style score, then renders full songs with vocals and accompaniment.