notesum.ai

Published at December 9

Continual Learning for Segment Anything Model Adaptation

cs.CV

Released Date: December 9, 2024

Authors: Jinglong Yang¹, Yichen Wu², Jun Cen³, Wenjian Huang⁴, Hong Wang⁴, Jianguo Zhang¹

Aff.: ¹Southern University of Science and Technology; ²City University of Hong Kong; ³Hong Kong University of Science and Technology; ⁴Tencent YouTu Lab

Arxiv: http://arxiv.org/pdf/2412.06418v1

Method	IoU			BIoU
Method	Last-IoU $\uparrow$	Avg-IoU $\uparrow$	FF-IoU $\downarrow$	Last-BIoU $\uparrow$	Avg-BIoU $\uparrow$	FF-BIoU $\downarrow$
Joint Training (Upper Bound)	76.3 $\pm$ 0.31	-	-	66.9 $\pm$ 0.26	-	-
SAM	69.0	-	-	58.9	-	-
LwF	73.3 $\pm$ 0.72	72.0 $\pm$ 0.68	2.0 $\pm$ 0.39	63.7 $\pm$ 0.78	69.1 $\pm$ 0.69	2.3 $\pm$ 0.31
EWC	72.9 $\pm$ 1.25	72.3 $\pm$ 0.97	3.5 $\pm$ 0.56	63.2 $\pm$ 1.01	68.9 $\pm$ 0.82	3.9 $\pm$ 0.48
ER	73.0 $\pm$ 0.93	73.2 $\pm$ 0.71	2.9 $\pm$ 0.39	62.7 $\pm$ 0.89	69.8 $\pm$ 0.63	3.4 $\pm$ 0.30
L2P	63.1 $\pm$ 1.15	56.5 $\pm$ 1.49	2.0 $\pm$ 0.43	52.5 $\pm$ 1.24	53.2 $\pm$ 1.31	3.3 $\pm$ 0.56
Dual-Prompt	64.9 $\pm$ 1.43	64.5 $\pm$ 1.63	2.7 $\pm$ 0.40	55.0 $\pm$ 1.71	61.1 $\pm$ 1.56	2.6 $\pm$ 0.39
Coda-Prompt	65.9 $\pm$ 1.45	63.8 $\pm$ 1.62	2.8 $\pm$ 0.37	55.3 $\pm$ 1.28	60.3 $\pm$ 1.43	2.1 $\pm$ 0.35
Decoder-Tuning	27.3 $\pm$ 1.69	46.9 $\pm$ 1.37	45.0 $\pm$ 1.82	19.4 $\pm$ 1.57	43.3 $\pm$ 1.48	43.7 $\pm$ 1.66
Decoder-Tuning + MoDA	55.8 $\pm$ 0.58 ( $\uparrow 28.5$ )	51.2 $\pm$ 0.52 ( $\uparrow 4.3$ )	0.33 $\pm$ 0.12 ( $\downarrow 44.7$ )	44.8 $\pm$ 0.49 ( $\uparrow 25.4$ )	47.4 $\pm$ 0.43 ( $\uparrow 4.1$ )	0.27 $\pm$ 0.09 ( $\downarrow 43.4$ )
SAM-Adapter	67.4 $\pm$ 0.75	69.3 $\pm$ 0.63	8.2 $\pm$ 0.59	56.9 $\pm$ 0.81	65.8 $\pm$ 0.74	9.1 $\pm$ 0.65
SAM-Adapter + MoDA	74.3 $\pm$ 0.37 ( $\uparrow 6.9$ )	73.4 $\pm$ 0.45 ( $\uparrow 4.1$ )	0.36 $\pm$ 0.1 ( $\downarrow 7.8$ )	64.3 $\pm$ 0.41 ( $\uparrow 7.4$ )	69.9 $\pm$ 0.38 ( $\uparrow 4.1$ )	0.27 $\pm$ 0.11 ( $\downarrow 8.8$ )
HQ-SAM	68.7 $\pm$ 0.59	70.9 $\pm$ 0.37	4.9 $\pm$ 0.44	58.6 $\pm$ 0.23	67.9 $\pm$ 0.35	9.3 $\pm$ 0.40
HQ-SAM + MoDA	75.2 $\pm$ 0.42 ( $\uparrow 6.5$ )	74.9 $\pm$ 0.61 ( $\uparrow$ 4.0)	0.28 $\pm$ 0.09 ( $\downarrow 4.6$ )	65.3 $\pm$ 0.37 ( $\uparrow 6.7$ )	71.6 $\pm$ 0.50 ( $\uparrow 3.7$ )	0.14 $\pm$ 0.07 ( $\downarrow 9.2$ )