notesum.ai

Published at November 27

Proactive Gradient Conflict Mitigation in Multi-Task Learning: A Sparse Training Perspective

cs.LG

cs.AI

cs.CV

Released Date: November 27, 2024

Authors: Zhi Zhang¹, Jiayi Shen¹, Congfeng Cao¹, Gaole Dai², Shiji Zhou³, Qizhe Zhang², Shanghang Zhang², Ekaterina Shutova¹

Aff.: ¹ILLC, University of Amsterdam, Netherlands; ²State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University, China; ³Department of Automation, Tsinghua University, China

Arxiv: http://arxiv.org/abs/2411.18615v1

Methods	Segmentation		Depth		Surface Normal					$\mathbf{\Delta m\%}\downarrow$
	mIoU $\uparrow$	Pix Acc $\uparrow$	Abs Err $\downarrow$	Rel Err $\downarrow$	Angle Distance $\downarrow$		Within $t^{\circ}$ $\uparrow$
	mIoU $\uparrow$	Pix Acc $\uparrow$	Abs Err $\downarrow$	Rel Err $\downarrow$	Mean	Median	11.25	22.5	30
STL	$58.62$	$79.20$	$0.3810$	$0.1553$	$19.29$	$12.64$	$46.37$	$72.19$	$80.73$	$-$
Joint Train	$59.09$	$79.61$	$0.3348$	$0.1360$	$22.34$	$16.33$	$35.46$	$64.02$	$75.20$	$6.763$
w/ ST	$60.03$	$79.96$	$0.3320$	$0.1353$	$21.98$	$15.92$	$36.69$	$64.92$	$75.82$	$5.314$
PCGrad	$59.18$	$80.12$	$0.3258$	$0.1323$	$21.81$	$15.72$	$36.92$	$65.49$	$76.26$	$4.584$
w/ ST	$59.37$	80.33	$0.3272$	$0.1330$	$21.53$	$15.39$	$38.02$	$66.09$	$76.71$	$3.741$
CAGrad	$59.78$	$80.16$	0.3215	0.1305	$19.92$	$13.40$	$43.87$	$70.47$	$79.59$	$-1.816$
w/ ST	$60.33$	$80.20$	$0.3232$	$0.1306$	$19.74$	$13.20$	$44.42$	$71.04$	$80.02$	$\mathbb{-2.423}$
GradDrop	$59.02$	$79.80$	$0.3283$	$0.1321$	$22.03$	$15.95$	$36.42$	$64.90$	$75.79$	$5.323$
w/ ST	$59.74$	$80.32$	$0.3278$	$0.1322$	$21.81$	$15.63$	$37.40$	$65.50$	$76.15$	$4.329$
MGDA	$37.43$	$67.58$	$0.4427$	$0.1810$	$19.23$	12.61	$46.43$	72.35	80.87	$9.162$
w/ ST	$41.60$	$69.96$	$0.4414$	$0.1778$	19.22	12.61	46.44	$72.29$	$80.80$	$7.791$
IMTL-G	60.64	$80.29$	$0.3324$	$0.1348$	$19.85$	$13.37$	$43.92$	$70.78$	$79.9$	$-1.537$
w/ ST	$60.35$	$80.18$	$0.3347$	$0.1350$	$19.65$	$13.15$	$44.66$	$71.25$	$80.20$	$-1.955$
NashMTL	$59.42$	$80.20$	$0.3303$	$0.1341$	$19.90$	$13.39$	$43.86$	$70.65$	$79.72$	$-1.295$
w/ ST	$59.36$	$79.98$	$0.3278$	$0.1323$	$19.63$	$13.02$	$45.06$	$71.31$	$80.15$	$-2.384$