Open-source dense 3D point tracker that repurposes the Wan2.1 video diffusion transformer to track pixels in one pass.
Updated EU status checked Last shown
Sources: the vendor's own pages (linked above) and creator videos at the linked timestamps. EU status is information, not legal advice; see how we check.
TrackCraft3R is a research model from the KAIST CVLab group (cvlab-kaist) that repurposes a pre-trained video diffusion transformer, Wan2.1-T2V-1.3B, as a single-pass dense 3D tracker. Given a monocular video plus its predicted depth and camera parameters, it predicts dense 3D trajectories for pixels in a single forward pass, together with visibility.
Training uses a two-stage recipe: first DiT LoRA with input/output projections, then additionally unfreezing the VAE, with separate RGB and pointmap encoders and residual-track and visibility decoders. It is trained on four synthetic datasets (Kubric, Dynamic Replica, PointOdyssey, TartanAir). The repository includes evaluation scripts for long-video (interleaved) and large-motion (stride) inference. A creator review reports it outperforming Motion Tracker and Any4D on benchmarks with lower memory use; this is not independently verified here.
The code is released under Apache-2.0 on GitHub, and checkpoints are on Hugging Face. Running your own video requires a preprocessing step with Depth-Anything-V3 or ViPE to get depth and camera. Training was tested on 8 NVIDIA H200 GPUs, so it targets researchers with GPU access. Note that the base Wan2.1 weights carry their own license terms.
Watch the moment: the exact second top creators show it. Videos load only when you press play, via youtube-nocookie.com.
Repurposes a video diffusion model to track every pixel in 3D through a video. It beats Motion Tracker and Any4D on benchmarks, handles long videos and large motion, and uses less memory. Code released.
Closest neighbours by meaning, not just by category.
Model · Nvidia
Feed-forward model that repairs lifted 2D tracks and depth into accurate dense 3D trajectories from monocular video.
Research
Post-training method that refines feed-forward 3D reconstruction models (Pi3, DA3, VGGT) on unlabeled video.
Model · ByteDance
ByteDance's open-weights one-step diffusion transformer for restoring and upscaling video and images.
Model
Research model that predicts layered, pixel-aligned 3D point clouds, including hidden geometry, from one image or video.
Model
3D-aware 360° video diffusion model that generates panoramic videos along user-defined camera paths from sparse panoramas.
Research
Research framework for globally consistent 3D reconstruction from long RGB videos using chunked priors and test-time adaptation.
