notesum.ai

Published at December 5

QUEEN: QUantized Efficient ENcoding of Dynamic Gaussians for Streaming Free-viewpoint Videos

cs.CV

cs.AI

Released Date: December 5, 2024

Authors: Sharath Girish¹, Tianye Li², Amrita Mazumdar², Abhinav Shrivastava¹, David Luebke², Shalini De Mello²

Aff.: ¹University of Maryland; ²NVIDIA

Arxiv: http://arxiv.org/pdf/2412.04469v1

Refer to caption

Neural 3D Video (N3DV) dataset
Category	Method	PSNR (dB) $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	Storage (MB) $\downarrow$	Training (sec) $\downarrow$	Rendering (FPS) $\uparrow$
Offline	NeRFPlayer [68]	30.69	0.932	0.209	17.10	72	0.05
	HyperReel [1]	31.10	0.928	-	1.20	104	2.00
	SpaceTime [42]	32.05	0.948	-	0.67	20	140
Online	StreamRF [37]	30.68	-	-	31.4	15	8.3
	TeTriRF [84]	30.43	0.906	0.248	0.06	39	4
	3DGStream [69]	31.67	-	-	7.83	12	215
	3DGStream* [69]	31.58	0.941	0.140	7.80	8.5	261
	QUEEN-s (ours)	31.89	0.945	0.139	0.68	4.65	345
	QUEEN-m (ours)	32.03	0.946	0.137	0.69	5.96	321
	QUEEN-l (ours)	32.19	0.946	0.136	0.75	7.9	248
Immersive dataset
Category	Method	PSNR (dB) $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	Storage (MB) $\downarrow$	Training (sec) $\downarrow$	Rendering (FPS) $\uparrow$
Offline	NeRFPlayer [68]	25.8	0.848	0.329	17.1	${\sim}$ 72	0.12
	HyperReel [1]	28.8	0.874	-	1.2	${\sim}$ 108	4
	SpaceTime [42]	29.2	0.916	-	1.2	${\sim}$ 72	99
Online	3DGStream* [69]	25.18	0.876	0.255	8.83	32.4	221
	QUEEN-l (ours)	29.22	0.915	0.208	1.79	19.7	183