Research method that recovers 6-DoF object and camera pose from monocular video via proxy-video translation and PnP.
Updated EU status checked Last shown
Sources: the vendor's own pages (linked above) and creator videos at the linked timestamps. EU status is information, not legal advice; see how we check.
ProxyPose is a research method for 6-DoF pose tracking from monocular video, developed by Ruihang Zhang, Felix Taubner, Pooja Ravi, Kiriakos N. Kutulakos and David B. Lindell (Toronto Computational Imaging Group). It is slated for NeurIPS 2026. Given a source video and a single query pixel, it uses a LoRA-finetuned video generative model to translate the input into a "proxy video" in which a colored cube follows the same local rigid-body motion as the queried point.
The pose trajectory is then recovered from the proxy video with learning-free geometry: contour fitting and Perspective-n-Point (PnP) optimization. Creator coverage reports it is built on Wan 2.1. The authors state it handles transparent, reflective and heavily occluded objects better than prior methods, and report state-of-the-art results on HO3D, YCBInEOAT and their ProxyPose Synthetic Benchmark even with a single query. For rigid objects, multi-query bundle adjustment can improve results further.
Beyond object pose, it can track camera pose by following a static point (including cases where COLMAP fails), track human faces despite training only on rigid objects, and be applied without extra training to event-camera and single-photon (SPAD) video. Stated limitations: pose drift on reflective or textureless surfaces under complex motion, inconsistency where tracking is ill-defined (e.g. fluid surfaces), and blurred proxy frames under fast motion due to the video model's VAE. The page thanks contributors for help with an open-source code release, but no repository link was provided.
Watch the moment: the exact second top creators show it. Videos load only when you press play, via youtube-nocookie.com.
Tracks full 3D position and rotation of objects in a video by using a video model to replace the object with a proxy shape, then using geometry to compute motion. Built on Wan 2.1, works on transparent and reflective objects.
Closest neighbours by meaning, not just by category.
Research
Research framework for globally consistent 3D reconstruction from long RGB videos using chunked priors and test-time adaptation.
Research
Single-step GRPO training framework for aligning video diffusion models with human preferences, tested on Wan2.1 1.3B–14B.
Model
Monocular video motion capture that combines flow matching with physics-based RL for plausible 3D human motion.
Research
Post-training method that refines feed-forward 3D reconstruction models (Pi3, DA3, VGGT) on unlabeled video.
Research
Research method that reconstructs complete 4D (3D plus time) scenes from a single monocular video using Gaussian Splatting.
Research
Multi-view video diffusion model that turns monocular or sparse videos of dynamic humans into dense multi-view videos for 4D Gaussian splats.