VideoMDM

VideoMDM is a research framework from Amir Mann, Gal Michael Harari, Merav Keidar and Or Litany, accepted at NeurIPS 2026. It trains a diffusion-based 3D human motion prior using only 2D keypoints from monocular videos, with no 3D ground truth or motion-capture data.

A pretrained 2D-to-3D lifter provides approximate 3D pose sequences as a noisy teacher. The model diffuses and denoises these in 3D, but supervision is applied in 2D by reprojecting the prediction and comparing it to accurate keypoints. The authors state that a depth-weighted 2D reprojection loss is, under mild assumptions, equivalent in expectation to direct 3D supervision. They also adapt velocity-consistency and representation-alignment regularizers to the 2D setting.

On HumanML3D it reports FID 0.88 versus 0.54 for the fully 3D-supervised MDM. On real video datasets Fit3D (fitness videos, with WHAM as the lifter) and NBA, the authors report that its motions are consistently preferred by humans. A YouTube creator reports the code is released under MIT; no repository link was found in the provided pages. It is a research project, not a hosted product.