Lucid Lyra

LucidLyra is a research framework from NVIDIA and academic collaborators for feed-forward generative 3D scene reconstruction. From a single image, a frozen camera-controlled video diffusion model generates multi-view latents. A Gaussian decoder then predicts 3D Gaussians encoding geometry, appearance, open-vocabulary semantics and class-agnostic instance embeddings in one pass, without per-scene optimization.\n\nThe decoder is trained by distilling knowledge from the video backbone and frozen 2D vision foundation models (SAM/SAM2, SigLIP2, C-RADIO features), using the backbone's generated videos. This requires neither manual 3D annotations nor captured multi-view data. The method also extends to monocular video input, producing structured dynamic (4D) scenes. The project page shows applications in robot simulation, 3D scene editing and 3D asset mining, plus evaluations on synthetic benchmarks and Waymo real-image data.\n\nThe project is currently a research release described by a paper (arXiv:2610.40495) and project page. Code and weights are not yet published; a creator review says code is coming soon. Availability and licensing are therefore unconfirmed.