notesum.ai

Published at December 6

REGENT: A Retrieval-Augmented Generalist Agent That Can Act In-Context in New Environments

cs.AI

Released Date: December 6, 2024

Authors: Kaustubh Sridhar¹, Souradeep Dutta¹, Dinesh Jayaraman¹, Insup Lee¹

Aff.: ¹University of Pennsylvania

Arxiv: http://arxiv.org/pdf/2412.04759v1

Env	Num	R&P	REGENT	REGENT	JAT/Gato	JAT/Gato	JAT/Gato	JAT/Gato	JAT/Gato	JAT/Gato	JAT/Gato	JAT/Gato	Train	DRIL
	demos			Finetuned		+ RAG	Finetuned	PEFT	(All Data)	(All Data)	(All Data)	(All Data)	From
								with		+ RAG	Finetuned	PEFT	Scratch
								IA3				with IA3
metaworld-bin-picking	25	0.33	0.31 $\pm$ 0.142	0.3 $\pm$ 0.025	0.0	0.01	0.0	0.0 $\pm$ 0.005	0.0	0.01	0.0	0.01 $\pm$ 0.0	0.0	-
metaworld-bin-picking	50	0.4	0.55 $\pm$ 0.031	0.6 $\pm$ 0.049	0.0	0.04	0.0	0.01 $\pm$ 0.005	0.0	0.01	0.0	0.01 $\pm$ 0.0	0.0	-
metaworld-bin-picking	75	0.34	0.59 $\pm$ 0.097	0.69 $\pm$ 0.021	0.0	0.08	0.01	0.01 $\pm$ 0.005	0.0	0.01	0.01	0.01 $\pm$ 0.0	0.0	-
metaworld-bin-picking	100	0.33	0.67 $\pm$ 0.051	0.73 $\pm$ 0.005	0.0	0.06	0.0	0.01 $\pm$ 0.005	0.0	0.02	0.01	0.01 $\pm$ 0.0	0.0	-
metaworld-box-close	25	0.58	0.73 $\pm$ 0.123	0.81 $\pm$ 0.09	0.0	0.0	0.0	0.0 $\pm$ 0.0	0.0	0.09	0.0	0.0 $\pm$ 0.0	0.01	-
metaworld-box-close	50	0.69	0.82 $\pm$ 0.142	0.86 $\pm$ 0.102	0.0	0.2	0.0	0.0 $\pm$ 0.0	0.0	0.2	0.0	0.0 $\pm$ 0.0	0.0	-
metaworld-box-close	75	0.68	0.84 $\pm$ 0.026	0.99 $\pm$ 0.017	0.0	0.4	0.0	0.0 $\pm$ 0.0	0.0	0.23	0.0	0.0 $\pm$ 0.0	0.0	-
metaworld-box-close	100	0.81	0.93 $\pm$ 0.067	0.98 $\pm$ 0.005	0.0	0.56	0.0	0.0 $\pm$ 0.0	0.0	0.31	0.0	0.0 $\pm$ 0.0	0.0	-
metaworld-door-lock	25	0.58	0.73 $\pm$ 0.019	0.8 $\pm$ 0.012	0.13	0.33	0.12	0.12 $\pm$ 0.012	0.24	0.18	0.31	0.23 $\pm$ 0.039	0.0	-
metaworld-door-lock	50	0.55	0.81 $\pm$ 0.047	0.82 $\pm$ 0.014	0.13	0.34	0.15	0.09 $\pm$ 0.021	0.24	0.24	0.25	0.29 $\pm$ 0.028	0.03	-
metaworld-door-lock	75	0.63	0.85 $\pm$ 0.028	0.84 $\pm$ 0.022	0.13	0.39	0.17	0.14 $\pm$ 0.029	0.24	0.31	0.27	0.4 $\pm$ 0.026	0.01	-
metaworld-door-lock	100	0.64	0.85 $\pm$ 0.028	0.85 $\pm$ 0.0	0.13	0.48	0.17	0.15 $\pm$ 0.029	0.24	0.38	0.28	0.42 $\pm$ 0.019	0.03	-
metaworld-door-unlock	25	0.53	0.65 $\pm$ 0.054	0.72 $\pm$ 0.037	0.01	0.36	0.03	0.22 $\pm$ 0.057	0.06	0.31	0.07	0.15 $\pm$ 0.036	0.0	-
metaworld-door-unlock	50	0.55	0.81 $\pm$ 0.049	0.84 $\pm$ 0.031	0.01	0.47	0.03	0.23 $\pm$ 0.08	0.06	0.42	0.07	0.25 $\pm$ 0.083	0.01	-
metaworld-door-unlock	75	0.78	0.88 $\pm$ 0.008	0.92 $\pm$ 0.005	0.01	0.41	0.02	0.2 $\pm$ 0.022	0.06	0.44	0.05	0.3 $\pm$ 0.074	0.0	-
metaworld-door-unlock	100	0.86	0.9 $\pm$ 0.017	0.93 $\pm$ 0.005	0.01	0.45	0.03	0.16 $\pm$ 0.031	0.06	0.5	0.08	0.24 $\pm$ 0.094	0.03	-
metaworld-hand-insert	25	0.27	0.62 $\pm$ 0.022	0.54 $\pm$ 0.022	0.01	0.07	0.07	0.1 $\pm$ 0.019	0.01	0.06	0.05	0.29 $\pm$ 0.034	0.0	-
metaworld-hand-insert	50	0.54	0.68 $\pm$ 0.031	0.69 $\pm$ 0.012	0.01	0.13	0.04	0.16 $\pm$ 0.031	0.01	0.1	0.06	0.58 $\pm$ 0.012	0.0	-
metaworld-hand-insert	75	0.56	0.78 $\pm$ 0.028	0.79 $\pm$ 0.021	0.01	0.22	0.04	0.16 $\pm$ 0.022	0.01	0.1	0.06	0.63 $\pm$ 0.017	0.0	-
metaworld-hand-insert	100	0.69	0.82 $\pm$ 0.042	0.83 $\pm$ 0.0	0.01	0.23	0.02	0.15 $\pm$ 0.054	0.01	0.14	0.03	0.61 $\pm$ 0.065	0.01	-
atari-pong	7 (11599)	0.08	0.08 $\pm$ 0.0	0.17 $\pm$ 0.042	0.0	0.0	0.0	0.0 $\pm$ 0.0	0.0	0.0	0.0	0.0 $\pm$ 0.0	0.0	0.0 $\pm$ 0.0
atari-pong	10 (16533)	0.18	0.2 $\pm$ 0.0	0.23 $\pm$ 0.0	0.0	0.0	0.0