notesum.ai

Published at November 4

Constrained Human-AI Cooperation: An Inclusive Embodied Social Intelligence Challenge

cs.AI

cs.HC

cs.RO

Released Date: November 4, 2024

Authors: Weihua Du¹, Qiushi Lyu², Jiaming Shan³, Zhenting Qi⁴, Hongxin Zhang⁵, Sunli Chen⁵, Andi Peng⁶, Tianmin Shu⁷, Kwonjoon Lee⁸, Behzad Dariush⁸, Chuang Gan⁵

Aff.: ¹Carnegie Mellon University; ²Peking University; ³University of California, Santa Barbara; ⁴Harvard University; ⁵University of Massachusetts Amherst; ⁶MIT; ⁷Johns Hopkins University; ⁸Honda Research Institute USA

Arxiv: http://arxiv.org/abs/2411.01796v1

Indoor
Helper Agent	No Constraint		High Target		High Container		High Goalplace
Helper Agent	TR(EI) $\uparrow$	IA $\uparrow$	TR(EI) $\uparrow$	IA $\uparrow$	TR(EI) $\uparrow$	IA $\uparrow$	TR(EI) $\uparrow$	IA $\uparrow$
w/o	0.53	/	0.30	/	0.37	/	0.28	/
Random	0.52(-0.02)	0.24	0.27(-0.05)	0.29	0.36(0.00)	0.25	0.33(0.10)	0.14
RHP	0.64(0.15)	0.15	0.35(0.11)	0.29	0.45(0.19)	0.21	0.35(0.18)	0.21
VLM (GPT-4o)	0.63(0.14)	0.24	0.33(0.06)	0.32	0.43(0.12)	0.40	0.26(-0.20)	0.33
LLM (GPT-4) + BM	0.65(0.17)	0.25	0.38(0.19)	0.29	0.49(0.24)	0.30	0.36(0.23)	0.35
Oracle	0.77(0.31)	0.88	0.49(0.37)	0.91	0.69(0.47)	0.91	0.61(0.56)	0.90
Indoor					Outdoor
Helper Agent	Low Target		Obstacle		Shopping			Furniture
Helper Agent	TR(EI) $\uparrow$	IA $\uparrow$	TR(EI) $\uparrow$	IA $\uparrow$	TR(EI) $\uparrow$	IA $\uparrow$	ER $\downarrow$	TR(EI) $\uparrow$
w/o	0.51	/	0.07	/	0.37	/	/	0.17
Random	0.50(-0.01)	0.31	0.21(0.56)	0.24	0.39(0.05)	0.34	0.32	0.48(0.68)
RHP	0.66(0.23)	0.28	0.44(0.77)	0.17	0.49(0.22)	0.44	0.30	0.65(0.72)
VLM (GPT-4o)	0.69(0.26)	0.46	0.40(0.86)	0.35	0.50(0.25)	0.72	0.39	0.70(0.78)
LLM (GPT-4) + BM	0.70(0.27)	0.43	0.42(0.89)	0.47	0.58(0.33)	0.74	0.38	0.69(0.77)
Oracle	0.82(0.38)	0.91	0.60(0.87)	0.82	0.61(0.39)	0.87	0.17	0.76(0.80)