Monocular Dynamic Gaussian Splatting is Fast and Brittle but Smooth Motion Helps



*Brown University    †NVIDIA Research    ‡Stanford University   

Dataset Paper Code

Dynamic Gaussians typically originate from static 3DGS by adding a Dynamics module.

Abstract


Gaussian splatting methods are emerging as a popular approach for converting multi-view image data into scene representations that allow view synthesis. In particular, there is interest in enabling view synthesis for dynamic scenes using only monocular input data---an ill-posed and challenging problem. The fast pace of work in this area has produced multiple simultaneous papers that claim to work best, which cannot all be true. In this work, we organize, benchmark, and analyze many Gaussian-splatting-based methods, providing apples-to-apples comparisons that prior works have lacked. We use multiple existing datasets and a new instructive synthetic dataset designed to isolate factors that affect reconstruction quality.



Qualitative Results



Instructive Dataset Results


SlidingCube




Click to select different cube motion range {0, 5, 10} and camera baseline range {1, 3, 5, 10, 20}.


RotatingCube




Click to select different cube motion range {0, 5, 10} and camera baseline range {1, 3, 5, 10, 20}.



Existing Dataset Results


DNeRF


bouncingballs standup trex

Click to select different scenes.


HyperNeRF


vrig-peel-banana torchocolate espresso

Click to select different scenes.


Nerfies


toby-sit curls tail

Click to select different scenes.


iPhone


apple mochi-high-five paper-windmill

Click to select different scenes.


NeRF-DS


as sieve plate

Click to select different scenes.