notesum.ai

Published at November 11

Grounding Video Models to Actions through Goal Conditioned Exploration

cs.RO

cs.AI

cs.CV

cs.LG

Released Date: November 11, 2024

Authors: Yunhao Luo¹, Yilun Du²

Aff.: ¹Georgia Tech; ²Brown

Arxiv: http://arxiv.org/abs/2411.07223v1

Refer to caption

	put-red-mug-left	put-red-mug-right	put-white-mug-left	put-Y/W-mug-right	Overall
BC*	8.8 $\pm\mathtt{5.3}$	15.2 $\pm\mathtt{7.8}$	32.0 $\pm\mathtt{12.9}$	21.6 $\pm\mathtt{12.3}$	19.4 $\pm\mathtt{9.6}$
GCBC*	2.4 $\pm\mathtt{2.0}$	0.8 $\pm\mathtt{1.6}$	16.0 $\pm\mathtt{7.2}$	7.2 $\pm\mathtt{5.3}$	6.6 $\pm\mathtt{4.0}$
DP BC*	33.6 $\pm\mathtt{3.2}$	33.6 $\pm\mathtt{8.2}$	59.2 $\pm\mathtt{7.8}$	57.6 $\pm\mathtt{5.4}$	46.0 $\pm\mathtt{6.2}$
DP GCBC*	24.8 $\pm\mathtt{4.7}$	22.4 $\pm\mathtt{7.4}$	16.0 $\pm\mathtt{8.8}$	3.2 $\pm\mathtt{3.0}$	16.6 $\pm\mathtt{6.0}$
SuSIE*	18.4 $\pm\mathtt{2.0}$	32.0 $\pm\mathtt{8.4}$	43.2 $\pm\mathtt{4.7}$	25.6 $\pm\mathtt{11.5}$	29.8 $\pm\mathtt{6.6}$
AVDC	0.0 $\pm\mathtt{0.0}$	0.0 $\pm\mathtt{0.0}$	0.0 $\pm\mathtt{0.0}$	0.0 $\pm\mathtt{0.0}$	0.0 $\pm\mathtt{0.0}$
Ours w/ SuSIE	23.2 $\pm\mathtt{3.0}$	60.0 $\pm\mathtt{6.7}$	68.8 $\pm\mathtt{4.7}$	67.2 $\pm\mathtt{8.9}$	54.8 $\pm\mathtt{5.8}$
Ours	38.4 $\pm\mathtt{15.3}$	40.8 $\pm\mathtt{7.8}$	51.2 $\pm\mathtt{3.9}$	38.4 $\pm\mathtt{8.6}$	42.2 $\pm\mathtt{8.9}$