notesum.ai

Published at December 9

Policy Agnostic RL: Offline RL and Online RL Fine-Tuning of Any Class and Backbone

cs.LG

cs.AI

Released Date: December 9, 2024

Authors: Max Sobol Mark¹, Tian Gao², Georgia Gabriela Sampaio², Mohan Kumar Srirama¹, Archit Sharma², Chelsea Finn², Aviral Kumar¹

Aff.: ¹Carnegie Mellon University; ²Stanford University

Arxiv: http://arxiv.org/pdf/2412.06685v1

Refer to caption

Kitchen (-v0)
Domain / Task	IDQL	DQL	DPPO	Cal-QL	PA-RL + Cal-QL (Ours)
CALVIN	19 $\to$ 35	19 $\to$ 22	13 $\to$ 18	6 $\to$ 36	28 $\to$ 61
complete	65 $\to$ 72	70 $\to$ 44	55 $\to$ 76	19 $\to$ 57	59 $\to$ 90
mixed	60 $\to$ 70	56 $\to$ 57	45 $\to$ 75	37 $\to$ 72	67 $\to$ 77
partial	70 $\to$ 90	56 $\to$ 46	38 $\to$ 69	59 $\to$ 84	78 $\to$ 94
Antmaze (-v2)
large-diverse	66 $\to$ 69	22 $\to$ 38	0 $\to$ 1	33 $\to$ 95	73 $\to$ 95
large-play	53 $\to$ 41	60 $\to$ 18	2 $\to$ 17	26 $\to$ 90	87 $\to$ 98
medium-diverse	83 $\to$ 86	14 $\to$ 70	43 $\to$ 95	75 $\to$ 98	88 $\to$ 98
medium-play	81 $\to$ 77	25 $\to$ 78	19 $\to$ 91	54 $\to$ 97	88 $\to$ 98
Aggregate	497 $\to$ 540	322 $\to$ 373	215 $\to$ 442	309 $\to$ 629	568 $\to$ 711