notesum.ai

Published at December 9

You See it, You Got it: Learning 3D Creation on Pose-Free Videos at Scale

cs.CV

Released Date: December 9, 2024

Authors: Baorui Ma¹, Huachen Gao¹, Haoge Deng¹, Zhengxiong Luo¹, Tiejun Huang¹, Lulu Tang¹, Xinlong Wang¹

Aff.: ¹Beijing Academy of Artificial Intelligence (BAAI)

Arxiv: http://arxiv.org/pdf/2412.06699v1

Refer to caption

Methods	Tanks-and-Temples [39]			RealEstate10K [117]			CO3D [69]
Single View	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
LucidDreamer [11]	13.11	0.314	0.485	15.24	0.545	0.357	13.90	0.412	0.473
ZeroNVS [71]	13.38	0.344	0.525	15.37	0.556	0.397	14.23	0.444	0.495
MotionCtrl [96]	14.31	0.405	0.436	16.30	0.596	0.363	16.16	0.515	0.418
ViewCrafter [112]	19.66	0.609	0.238	21.93	0.797	0.161	20.17	0.664	0.283
ViewCrafter* [112]	19.13	0.616	0.255	20.49	0.802	0.183	19.07	0.678	0.339
Ours	23.76	0.735	0.191	25.36	0.854	0.146	24.28	0.765	0.251
Sparse Views (3 Views)	LLFF [58]			DTU [33]			MipNeRF-360 [2]
Zip-NeRF^† [3]	17.23	0.574	0.373	9.18	0.601	0.383	12.77	0.271	0.705
MuRF [104]	21.34	0.722	0.245	21.31	0.885	0.127	-	-	-
FSGS [118]	20.31	0.652	0.288	17.34	0.818	0.169	-	-	-
BGGS [25]	21.44	0.751	0.168	20.71	0.862	0.111	-	-	-
ZeroNVS^† [71]	15.91	0.359	0.512	16.71	0.716	0.223	14.44	0.316	0.680
DepthSplat [105]	17.64	0.521	0.321	15.59	0.525	0.373	13.85	0.254	0.621
ReconFusion [99]	21.34	0.724	0.203	20.74	0.875	0.124	15.50	0.358	0.585
CAT3D [21]	21.58	0.731	0.181	22.02	0.844	0.121	16.62	0.377	0.515
Ours	23.23	0.768	0.135	28.04	0.884	0.073	17.35	0.442	0.422