notesum.ai

Published at May 10

e-COP : Episodic Constrained Optimization of Policies

NeurIPS

Released Date: May 10, 2024

Authors: Akhil Agnihotri¹, Rahul Jain², Deepak Ramachandran³, Sahil Singla³

Aff.: ¹University of Southern California; ²Google DeepMind and USC; ³Google DeepMind

Arxiv: https://openreview.net/pdf/e2087acb4d653ff52e076f87d71a30d1772bd69a.pdf

Refer to caption

Task		e-COP	FOCOPS [43]	PPO-L [30]	PCPO [39]	P3O [41]	CPO [3]	APPO [15]	IPO [26]
Humanoid	R	1652.5 $\pm$ 13.4	1734.1 $\pm$ 27.4	1431.2 $\pm$ 25.2	1602.3 $\pm$ 10.1	1669.4 $\pm$ 13.7	1465.1 $\pm$ 55.3	1488.2 $\pm$ 29.3	1578.6 $\pm$ 25.2
Humanoid	C (20.0)	17.3 $\pm$ 0.3	19.7 $\pm$ 0.6	18.8 $\pm$ 1.5	16.3 $\pm$ 1.4	20.1 $\pm$ 3.3	18.5 $\pm$ 2.9	20.0 $\pm$ 1.3	19.1 $\pm$ 2.5
PointCircle	R	110.5 $\pm$ 9.3	81.6 $\pm$ 8.4	57.2 $\pm$ 9.2	68.2 $\pm$ 9.1	89.1 $\pm$ 7.1	65.3 $\pm$ 5.3	91.2 $\pm$ 9.6	68.7 $\pm$ 15.2
PointCircle	C (10.0)	9.8 $\pm$ 0.9	10.0 $\pm$ 0.4	9.8 $\pm$ 0.5	9.9 $\pm$ 0.4	9.9 $\pm$ 0.3	9.5 $\pm$ 0.9	10.2 $\pm$ 0.6	9.3 $\pm$ 0.5
AntCircle	R	198.6 $\pm$ 7.4	161.9 $\pm$ 22.2	134.4 $\pm$ 10.3	168.3 $\pm$ 13.3	182.6 $\pm$ 18.7	127.1 $\pm$ 12.1	155.5 $\pm$ 19.4	149.3 $\pm$ 33.6
AntCircle	C (10.0)	9.8 $\pm$ 0.6	9.9 $\pm$ 0.5	9.6 $\pm$ 1.6	9.5 $\pm$ 0.6	9.8 $\pm$ 0.2	10.1 $\pm$ 0.7	10.0 $\pm$ 0.5	9.5 $\pm$ 1.0
PointReach	R	81.5 $\pm$ 10.2	65.1 $\pm$ 9.6	46.1 $\pm$ 14.8	73.2 $\pm$ 7.4	76.3 $\pm$ 6.4	89.2 $\pm$ 8.1	74.3 $\pm$ 6.7	49.1 $\pm$ 10.6
PointReach	C (25.0)	24.5 $\pm$ 6.1	24.8 $\pm$ 7.6	25.1 $\pm$ 6.1	24.9 $\pm$ 5.6	26.3 $\pm$ 6.9	33.3 $\pm$ 10.7	26.3 $\pm$ 8.1	24.7 $\pm$ 11.3
AntReach	R	70.8 $\pm$ 14.6	48.3 $\pm$ 5.6	54.2 $\pm$ 9.5	39.4 $\pm$ 5.3	73.6 $\pm$ 5.1	102.3 $\pm$ 7.1	61.5 $\pm$ 10.4	45.2 $\pm$ 13.3
AntReach	C (25.0)	24.2 $\pm$ 8.4	25.1 $\pm$ 11.9	21.9 $\pm$ 10.7	27.9 $\pm$ 12.2	24.8 $\pm$ 7.3	35.1 $\pm$ 10.9	24.5 $\pm$ 6.4	24.9 $\pm$ 9.2
	R	258.1 $\pm$ 33.1	215.4 $\pm$ 45.6	276.3 $\pm$ 57.9	226.5 $\pm$ 29.2	201.5 $\pm$ 39.2	178.1 $\pm$ 23.8	184.4 $\pm$ 21.5	229.4 $\pm$ 32.8
Grid	C (75.0)	71.3 $\pm$ 26.9	76.6 $\pm$ 29.8	71.8 $\pm$ 25.1	72.6 $\pm$ 16.5	79.3 $\pm$ 19.3	69.3 $\pm$ 19.8	79.5 $\pm$ 35.8	74.2 $\pm$ 24.6
Bottleneck	R	345.1 $\pm$ 52.6	251.3 $\pm$ 59.1	298.3 $\pm$ 71.2	264.2 $\pm$ 43.8	291.1 $\pm$ 26.7	388.1 $\pm$ 36.6	220.1 $\pm$ 30.1	279.3 $\pm$ 43.8
Bottleneck	C (50.0)	49.7 $\pm$ 15.1	46.6 $\pm$ 19.8	41.4 $\pm$ 17.6	49.8 $\pm$ 10.5	45.3 $\pm$ 8.2	54.3 $\pm$ 13.5	47.4 $\pm$ 12.3	48.2 $\pm$ 14.6
Navigation	R	217.6 $\pm$ 11.5		175.1 $\pm$ 3.7		153.5 $\pm$ 25.2		135.7 $\pm$ 19.2	164.1 $\pm$ 12.8
	C1 (10.0)	9.6 $\pm$ 1.5	n/a	9.9 $\pm$ 1.9	n/a	9.9 $\pm$ 1.7	n/a	9.9 $\pm$ 2.1	10.0 $\pm$ 0.5
	C2 (25.0)	23.7 $\pm$ 4.1		22.3 $\pm$ 2.1		24.5 $\pm$ 4.1		23.9 $\pm$ 3.8	24.6 $\pm$ 3.1