notesum.ai

Published at December 9

ZeroKey: Point-Level Reasoning and Zero-Shot 3D Keypoint Detection from Large Language Models

cs.CV

Released Date: December 9, 2024

Authors: Bingchen Gong¹, Diego Gomez¹, Abdullah Hamdi², Abdelrahman Eldesokey³, Ahmed Abdelreheem³, Peter Wonka³, Maks Ovsjanikov¹

Aff.: ¹Ecole Polytechnique; ²Visual Geometry Group, University of Oxford; ³King Abdullah University of Science and Technology (KAUST)

Arxiv: http://arxiv.org/pdf/2412.06292v1

[Uncaptioned image]

Method	IoU (in %)
Distance Thresholds	0.001	0.01	0.02	0.03	0.04	0.05	0.06	0.07	0.08	0.09	0.10
HARRIS-3D	0.15	0.76	2.19	3.96	6.16	8.88	11.91	15.13	18.63	22.10	25.69
SIFT-3D	0.29	1.05	2.62	4.38	6.65	9.42	12.38	15.65	19.39	22.71	26.15
ISS	0.49	1.10	2.69	4.67	7.27	10.14	13.29	16.56	20.02	23.58	27.20
USIP	0.83	1.70	3.25	5.24	8.00	11.76	15.98	20.63	25.56	30.20	33.67
D3FEAT	2.36	3.86	7.27	10.37	13.37	17.12	21.41	26.12	30.39	34.82	38.41
UKPGAN	3.95	6.54	12.27	17.86	22.42	26.55	30.28	34.32	38.46	42.65	46.49
FPS	6.34	10.55	17.68	23.18	26.91	30.27	33.57	36.85	40.47	44.42	48.38
FSKD	7.00	7.94	11.17	16.74	23.99	31.14	38.14	43.97	49.30	53.62	57.03
B2-3D (62 views)	12.41	20.29	35.55	46.25	52.74	57.72	61.61	64.19	66.56	68.55	70.57
B2-3D (26 views)	8.60	14.39	25.10	33.31	39.29	44.11	47.85	50.83	53.17	55.11	56.90
StablePoints [16]	3.66	5.80	9.94	13.32	16.58	19.91	23.48	27.26	31.28	34.89	38.22
RedCircle [40]	0.21	0.34	0.64	1.16	1.90	3.05	4.81	7.55	11.06	14.92	18.50
GPT-4o	0.18	0.48	1.20	2.61	4.19	6.04	8.48	10.89	14.03	17.03	20.73
CLIP-DINOiser [46]	0.73	1.41	3.00	4.94	7.31	9.80	12.66	15.52	18.52	21.76	25.56
ZeroKey (Ours)	3.69	5.84	10.91	17.58	23.72	29.81	35.52	40.82	45.72	50.24	54.64