notesum.ai

Published at November 29

Blurred LiDAR for Sharper 3D: Robust Handheld 3D Scanning with Diffuse LiDAR and RGB

eess.IV

cs.CV

cs.LG

Released Date: November 29, 2024

Authors: Nikhil Behari¹, Aaron Young¹, Siddharth Somasundaram¹, Tzofi Klinghoffer¹, Akshat Dave¹, Ramesh Raskar¹

Aff.: ¹Massachusetts Institute of Technology

Arxiv: http://arxiv.org/pdf/2411.19474v1

(a) Full Texture Datasets: Scenes with full texture on object and ground plane.
Method	Blender				Chair				Hotdog				Lego
Method	PSNR ↑	SSIM ↑	D.MAE ↓	N.MAE ↓	PSNR ↑	SSIM ↑	D.MAE ↓	N.MAE ↓	PSNR ↑	SSIM ↑	D.MAE ↓	N.MAE ↓	PSNR ↑	SSIM ↑	D.MAE ↓	N.MAE ↓
Surfels RGB Only	28.92	0.848	0.081	34.27	28.68	0.852	0.062	28.95	30.85	0.859	0.068	36.12	27.67	0.794	0.074	38.40
Surfels RGB w/ Mono. Depth	27.77	0.876	0.033	17.50	27.94	0.866	0.028	16.06	28.97	0.886	0.031	20.75	27.42	0.818	0.032	21.64
Surfels RGB w/ Sparse LiDAR	24.68	0.839	0.058	24.74	26.06	0.840	0.057	23.44	27.85	0.856	0.054	29.49	26.12	0.801	0.058	30.63
Ours (RGB + Diffuse LiDAR)	30.67	0.881	0.025	19.17	30.25	0.891	0.017	18.05	30.14	0.894	0.016	24.62	28.39	0.845	0.025	25.27
(b) Textured Object Datasets: Scenes with textured objects on completely textureless ground planes.
Surfels RGB Only	25.44	0.944	0.226	50.88	22.89	0.928	0.474	59.75	37.02	0.962	0.104	41.49	26.06	0.878	0.176	54.25
Surfels RGB w/ Mono. Depth	27.66	0.952	0.129	28.91	21.99	0.932	0.452	55.76	36.04	0.961	0.065	26.33	27.65	0.888	0.052	23.67
Surfels RGB w/ Sparse LiDAR	26.96	0.937	0.105	36.91	25.64	0.936	0.115	38.68	27.03	0.932	0.080	34.98	25.77	0.879	0.093	39.99
Ours (RGB + Diffuse LiDAR)	25.08	0.945	0.033	10.47	32.18	0.970	0.030	7.33	35.62	0.963	0.024	16.34	30.00	0.924	0.024	16.94
(c) Textured Plane Datasets: Scenes with completely textureless objects on textured ground planes.
Surfels RGB Only	26.34	0.848	0.090	37.53	23.69	0.841	0.089	36.90	29.30	0.879	0.101	44.77	25.86	0.858	0.091	42.49
Surfels RGB w/ Mono. Depth	26.34	0.883	0.036	19.27	23.76	0.867	0.049	20.61	30.41	0.921	0.050	27.08	24.95	0.889	0.055	28.24
Surfels RGB w/ Sparse LiDAR	25.36	0.867	0.057	23.95	24.92	0.842	0.063	31.37	28.33	0.894	0.045	29.28	24.69	0.861	0.067	31.06
Ours (RGB + Diffuse LiDAR)	23.72	0.820	0.045	21.13	25.70	0.841	0.037	22.80	28.97	0.858	0.034	22.32	26.60	0.850	0.046	25.69
(d) No Texture Datasets: Completely textureless scenes.
Surfels RGB Only	–	–	0.885	36.74	–	–	0.904	42.14	–	–	0.850	63.31	–	–	0.898	51.27
Surfels RGB w/ Mono. Depth	–	–	0.716	67.01	–	–	0.793	60.23	–	–	0.911	46.24	–	–	0.835	63.31
Surfels RGB w/ Sparse LiDAR	–	–	0.125	40.86	–	–	0.107	37.29	–	–	0.111	43.54	–	–	0.106	41.63
Ours (RGB + Diffuse LiDAR)	–	–	0.045	15.68	–	–	0.045	13.55	–	–	0.041	16.35	–	–	0.042	14.54