notesum.ai

Published at October 21

FlickerFusion: Intra-trajectory Domain Generalizing Multi-Agent RL

cs.CV

cs.AI

cs.CL

cs.LG

Released Date: October 21, 2024

Authors: Woosung Koh¹, Wonbeen Oh¹, Siyeol Kim¹, Suhin Shin¹, Hyeongjin Kim¹, Jaein Jang¹, Junghyun Lee², Se-Young Yun²

Aff.: ¹Yonsei University; ²KAIST AI

Arxiv: https://arxiv.org/abs/2410.15876v1

MPEv2 Environment	Spread		Repel		Tag
Method	OOD1	OOD2	OOD1	OOD2	OOD1	OOD2
FlickerFusion-Attention	-198.4 ${}_{\pm\text{17.0}}$	-230.1 ${}_{\pm\text{18.6}}$	845.6 ${}_{\pm\text{99.1}}$	1070.2 ${}_{\pm\text{215.4}}$	-2.1 ${}_{\pm\text{0.8}}$	-9.4 ${}_{\pm\text{3.8}}$
FlickerFusion-MLP	-183.7 ${}_{\pm\text{29.4}}$	-209.0 ${}_{\pm\text{25.4}}$	906.1 ${}_{\pm\text{42.3}}$	1189.0 ${}_{\pm\text{48.6}}$	-18.8 ${}_{\pm\text{24.3}}$	-33.7 ${}_{\pm\text{15.9}}$
ACORM (Hu et al., 2024)	-244.4 ${}_{\pm\text{24.1}}$	-330.3 ${}_{\pm\text{23.6}}$	153.6 ${}_{\pm\text{773.2}}$	98.1 ${}_{\pm\text{726.6}}$	-247.9 ${}_{\pm\text{174.1}}$	-578.2 ${}_{\pm\text{763.7}}$
ODIS (Zhang et al., 2023)	-257.1 ${}_{\pm\text{38.2}}$	-376.5 ${}_{\pm\text{84.6}}$	-3532.8 ${}_{\pm\text{2994.9}}$	-2982.1 ${}_{\pm\text{2074.4}}$	-1992.0 ${}_{\pm\text{2331.4}}$	-1723.4 ${}_{\pm\text{1817.3}}$
CAMA (Shao et al., 2023)	-262.2 ${}_{\pm\text{100.6}}$	-464.1 ${}_{\pm\text{244.4}}$	698.2 ${}_{\pm\text{126.2}}$	928.1 ${}_{\pm\text{198.9}}$	-29.6 ${}_{\pm\text{37.8}}$	-31.2 ${}_{\pm\text{28.4}}$
REFIL (Iqbal et al., 2021)	-205.0 ${}_{\pm\text{41.0}}$	-289.2 ${}_{\pm\text{29.2}}$	404.6 ${}_{\pm\text{351.2}}$	822.4 ${}_{\pm\text{148.4}}$	-9.3 ${}_{\pm\text{13.0}}$	-15.2 ${}_{\pm\text{9.8}}$
UPDeT (Hu et al., 2021)	-222.0 ${}_{\pm\text{80.4}}$	-273.9 ${}_{\pm\text{57.8}}$	-2645.2 ${}_{\pm\text{5006.1}}$	-1929.0 ${}_{\pm\text{3993.5}}$	-129.5 ${}_{\pm\text{50.9}}$	-128.8 ${}_{\pm\text{82.2}}$
Meta DotProd (Kedia et al., 2021)	-367.8 ${}_{\pm\text{95.1}}$	-555.9 ${}_{\pm\text{170.4}}$	-12538.6 ${}_{\pm\text{526.5}}$	-9365.7 ${}_{\pm\text{164.2}}$	-7611.4 ${}_{\pm\text{7279.8}}$	-7302.5 ${}_{\pm\text{7436.8}}$
DG-MAML (Wang et al., 2021a)	-217.3 ${}_{\pm\text{17.9}}$	-254.0 ${}_{\pm\text{15.3}}$	343.2 ${}_{\pm\text{438.0}}$	623.0 ${}_{\pm\text{420.4}}$	-39.2 ${}_{\pm\text{27.5}}$	-50.8 ${}_{\pm\text{30.8}}$
SMLDG (Li et al., 2020)	-258.2 ${}_{\pm\text{38.4}}$	-290.4 ${}_{\pm\text{32.8}}$	-1711.0 ${}_{\pm\text{2093.7}}$	-1211.7 ${}_{\pm\text{1868.4}}$	-210.6 ${}_{\pm\text{120.5}}$	-185.9 ${}_{\pm\text{148.8}}$
MLDG (Li et al., 2018)	-413.1 ${}_{\pm\text{47.1}}$	-610.7 ${}_{\pm\text{34.4}}$	-12763.4 ${}_{\pm\text{213.5}}$	-9711.1 ${}_{\pm\text{293.4}}$	-2625.9 ${}_{\pm\text{4899.6}}$	-2333.2 ${}_{\pm\text{4372.1}}$
QMIX-Attention (Iqbal et al., 2021)	-190.1 ${}_{\pm\text{8.6}}$	-251.5 ${}_{\pm\text{42.0}}$	719.0 ${}_{\pm\text{141.7}}$	927.4 ${}_{\pm\text{215.1}}$	-14.4 ${}_{\pm\text{9.8}}$	-26.6 ${}_{\pm\text{10.9}}$
QMIX-MLP (Rashid et al., 2018)	-209.1 ${}_{\pm\text{27.4}}$	-242.8 ${}_{\pm\text{37.3}}$	380.0 ${}_{\pm\text{195.0}}$	883.7 ${}_{\pm\text{225.6}}$	-41.6 ${}_{\pm\text{16.5}}$	-62.5 ${}_{\pm\text{36.4}}$
MPEv2 Environment	Guard		Adversary		Hunt
Method	OOD1	OOD2	OOD1	OOD2	OOD1	OOD2
FlickerFusion-Attention	-1258.3 ${}_{\pm\text{113.3}}$	-1160.0 ${}_{\pm\text{62.6}}$	60.9 ${}_{\pm\text{24.5}}$	9.9 ${}_{\pm\text{25.9}}$	-297.1 ${}_{\pm\text{20.2}}$	-337.5 ${}_{\pm\text{12.9}}$
FlickerFusion-MLP	-1127.2 ${}_{\pm\text{66.2}}$	-962.9 ${}_{\pm\text{49.1}}$	56.3 ${}_{\pm\text{20.1}}$	9.3 ${}_{\pm\text{13.2}}$	-278.5 ${}_{\pm\text{17.2 }}$	-305.3 ${}_{\pm\text{11.8}}$
ACORM (Hu et al., 2024)	-2074.3 ${}_{\pm\text{323.8}}$	-2049.6 ${}_{\pm\text{234.0}}$	56.1 ${}_{\pm\text{18.1}}$	13.1 ${}_{\pm\text{19.2}}$	-367.2 ${}_{\pm\text{59.0}}$	-397.3 ${}_{\pm\text{30.4}}$
ODIS (Zhang et al., 2023)	-1449.3 ${}_{\pm\text{80.1}}$	-1442.8 ${}_{\pm\text{278.6}}$	-56.8 ${}_{\pm\text{37.7}}$	-188.7 ${}_{\pm\text{105.2}}$	-667.5 ${}_{\pm\text{99.5}}$	-657.8 ${}_{\pm\text{209.7}}$
CAMA (Shao et al., 2023)	-5002.5 ${}_{\pm\text{2008.6}}$	-4656.9 ${}_{\pm\text{2252.9}}$	50.7 ${}_{\pm\text{11.1}}$	26.5 ${}_{\pm\text{14.5}}$	-1063.1 ${}_{\pm\text{397.3}}$	-1131.3 ${}_{\pm\text{565.5}}$
REFIL (Iqbal et al., 2021)	-1445.8 ${}_{\pm\text{196.8}}$	-1294.7 ${}_{\pm\text{88.6}}$	14.5 ${}_{\pm\text{16.6}}$	-11.7 ${}_{\pm\text{29.4}}$	-305.0 ${}_{\pm\text{18.6}}$	-347.9 ${}_{\pm\text{14.8}}$
UPDeT (Hu et al., 2021)	-2845.5 ${}_{\pm\text{2294.0}}$	-2637.0 ${}_{\pm\text{2393.4}}$	-64.6 ${}_{\pm\text{80.1}}$	-0.6 ${}_{\pm\text{59.6}}$	-340.8 ${}_{\pm\text{82.7}}$	-383.5 ${}_{\pm\text{110.5}}$
Meta DotProd (Kedia et al., 2021)	-7507.4 ${}_{\pm\text{550.0}}$	-7620.4 ${}_{\pm\text{504.2}}$	82.3 ${}_{\pm\text{15.5}}$	34.4 ${}_{\pm\text{20.8}}$	-956.0 ${}_{\pm\text{518.1}}$	-1087.9 ${}_{\pm\text{720.7}}$
DG-MAML (Wang et al., 2021a)	-1885.3 ${}_{\pm\text{187.3}}$	-1989.1 ${}_{\pm\text{288.6}}$	18.7 ${}_{\pm\text{34.7}}$	6.1 ${}_{\pm\text{40.6}}$	-384.1 ${}_{\pm\text{115.2}}$	-416 ${}_{\pm\text{123.4}}$
SMLDG (Li et al., 2020)	-2765.9 ${}_{\pm\text{743.8}}$	-2591.8 ${}_{\pm\text{1013.5}}$	-12.7 ${}_{\pm\text{39.9}}$	-134.8 ${}_{\pm\text{97.8}}$	-601.8 ${}_{\pm\text{283.9}}$	-623.0 ${}_{\pm\text{329}}$
MLDG (Li et al., 2018)	-10509.2 ${}_{\pm\text{662.2}}$	-9508.4 ${}_{\pm\text{580.8}}$	-76.2 ${}_{\pm\text{122.3}}$	-162.4 ${}_{\pm\text{201.6}}$	-1164.1 ${}_{\pm\text{511.0}}$	-1386.9 ${}_{\pm\text{724.8}}$
QMIX-Attention (Iqbal et al., 2021)	-1252.3 ${}_{\pm\text{75.0}}$	-1202.8 ${}_{\pm\text{137.9}}$	34.7 ${}_{\pm\text{20.1}}$	2.1 ${}_{\pm\text{17.0}}$	-305.3 ${}_{\pm\text{9.7}}$	-347.9 ${}_{\pm\text{15.7}}$
QMIX-MLP (Rashid et al., 2018)	-1464.6 ${}_{\pm\text{136.2}}$	-1325.1 ${}_{\pm\text{177.9}}$	17.3 ${}_{\pm\text{34.3}}$	-8.1 ${}_{\pm\text{13.2}}$	-337.8 ${}_{\pm\text{52.7}}$	-357.5 ${}_{\pm\text{14.4}}$