notesum.ai

Published at November 17

Mitigating Relative Over-Generalization in Multi-Agent Reinforcement Learning

cs.AI

stat.ML

Released Date: November 17, 2024

Authors: Ting Zhu¹, Yue Jin², Jeremie Houssineau³, Giovanni Montana⁴

Aff.: ¹Department of Statistics, University of Warwick, Coventry, UK; ²Warwick Manufacturing Group, University of Warwick, Coventry, UK; ³School of Physical & Mathematical Sciences, Nanyang Technological University, Singapore; ⁴Alan Turing Institute, London, UK

Arxiv: http://arxiv.org/abs/2411.11099v1

Setting	Setting	MMQ	IDDPG	HyDDPG	I2Q*	I2Q
DG	$N=2$	19.55 $\pm$ 0.16	14.67 $\pm$ 4.61	19.47 $\pm$ 0.17	17.84 $\pm$ 3.01	$14.62\pm$ 4.23
MPE Tasks	CN	-15.66 $\pm$ 1.75	-30.91 $\pm$ 12.78	-19.12 $\pm$ 1.56	-51.24 $\pm$ 12.01	-33.16 $\pm$ 11.82
MPE Tasks	CN+more penalty	-18.01 $\pm$ 2.24	-60.25 $\pm$ 0.07	-50.12 $\pm$ 13.24	-60.55 $\pm$ 0.36	-60.54 $\pm$ 0.70
	CN+HT	-24.25 $\pm$ 0.49	-40.34 $\pm$ 26.01	-30.93 $\pm$ 6.92	-56.04 $\pm$ 40.80	-35.95 $\pm$ 15.83
	CN+HA	-17.63 $\pm$ 4.12	-45.76 $\pm$ 13.99	-28.21 $\pm$ 9.71	-60.49 $\pm$ 0.15	-60.25 $\pm$ 0.03
	PP	-35.28 $\pm$ 0.40	-56.63 $\pm$ 0.11	-49.40 $\pm$ 7.18	-57.10 $\pm$ 0.33	-56.67 $\pm$ 0.14
	Sequential Task	-215.09 $\pm$ 59.97	-295.31 $\pm$ 9.75	-233.55 $\pm$ 53.21	-300.53 $\pm$ 0.45	-266.42 $\pm$ 43.37
Half-Cheetah	$2\times 3$	-134.09 $\pm$ 16.05	-163.81 $\pm$ 9.61	-152.66 $\pm$ 10.32	-135.65 $\pm$ 4.60	-140.94 $\pm$ 8.55