Research method that reconstructs 4D hand motion (MANO) from video using a fine-tuned video diffusion model.
Updated EU status checked Last shown
Sources: the vendor's own pages (linked above) and creator videos at the linked timestamps. EU status is information, not legal advice; see how we check.
ViDiHand is a research project for 4D hand motion recovery from monocular video. It adapts a video diffusion model: a VACE branch is fine-tuned with hand-overlay rendering while the base DiT stays frozen, producing a hand-aware video diffusion model. A lightweight dual-branch decoder then reads MANO pose, 2D joints and translation from a single intermediate VACE feature, and inference uses one VACE pass.
The project page reports comparisons on three egocentric benchmarks (ARCTIC, HOT3D, HOI4D; HOI4D is held out as cross-dataset) against methods such as HaMeR, WiLoR, Hamba, WildHands, Dyn-HaMR, HaWoR and OmniHands. It reports lower pose error and much lower jitter than these baselines, and the authors emphasise robustness to occlusion and temporal smoothness. Creators note possible uses in humanoid robot training and AR/VR.
The material provided shows a project page and a citation (arXiv 2606.30308, "The Surprising Effectiveness of Video Diffusion Models for Hand Motion Reconstruction"). Creator coverage says code is coming soon; no code, weights, or license were found, so it is not currently a usable product.
Watch the moment: the exact second top creators show it. Videos load only when you press play, via youtube-nocookie.com.
Reconstructs accurate 3D hand motion from video, even with occlusion, and outperforms competitors on benchmarks. Useful for humanoid robot training and AR/VR; code coming soon.
Closest neighbours by meaning, not just by category.
Research
Multi-view video diffusion model that turns monocular or sparse videos of dynamic humans into dense multi-view videos for 4D Gaussian splats.
Research
Research method that retargets human hand-object demonstrations to dexterous robot hands using one multi-reference RL policy.
Research
Diffusion framework that learns text-to-3D human motion generation from 2D poses extracted from monocular videos.
Research
Research method that reconstructs complete 4D (3D plus time) scenes from a single monocular video using Gaussian Splatting.
Model
Research method that recovers 6-DoF object and camera pose from monocular video via proxy-video translation and PnP.
Research · Nvidia
NVIDIA research framework that teaches dexterous robot hands manipulation from human demonstrations via contact wrenches.