Monocular Dynamic Gaussian Splatting is Fast and Brittle but Smooth Motion Helps



*Brown University    †NVIDIA Research    ‡Stanford University   

Dataset Arxiv Code

Dynamic Gaussians typically originate from static 3DGS by adding a Dynamics module.

Abstract


Gaussian splatting methods are emerging as a popular approach for converting multi-view image data into scene representations that allow view synthesis. In particular, there is interest in enabling view synthesis for dynamic scenes using only monocular input data---an ill-posed and challenging problem. The fast pace of work in this area has produced multiple simultaneous papers that claim to work best, which cannot all be true. In this work, we organize, benchmark, and analyze many Gaussian-splatting-based methods, providing apples-to-apples comparisons that prior works have lacked. We use multiple existing datasets and a new instructive synthetic dataset designed to isolate factors that affect reconstruction quality.

Bibtex


@misc{liang2024monocular, title={Monocular Dynamic Gaussian Splatting is Fast and Brittle but Smooth Motion Helps}, author={Yiqing Liang and Mikhail Okunev and Mikaela Angelina Uy and Runfeng Li and Leonidas Guibas and James Tompkin and Adam W. Harley}, year={2024}, eprint={2412.04457}, archivePrefix={arXiv}, primaryClass={cs.CV} }


Qualitative Results



Instructive Dataset Results


SlidingCube




Click to select different cube motion range {0, 5, 10} and camera baseline range {1, 3, 5, 10, 20}.


RotatingCube




Click to select different cube motion range {0, 5, 10} and camera baseline range {1, 3, 5, 10, 20}.



Existing Dataset Results


DNeRF


bouncingballs standup trex

Click to select different scenes.


HyperNeRF


vrig-peel-banana torchocolate espresso

Click to select different scenes.


Nerfies


toby-sit curls tail

Click to select different scenes.


iPhone


apple mochi-high-five paper-windmill

Click to select different scenes.


NeRF-DS


as sieve plate

Click to select different scenes.