notesum.ai

Published at October 29

Effective Guidance for Model Attention with Simple Yes-no Annotations

cs.LG

cs.AI

stat.ML

Released Date: October 29, 2024

Authors: Seongmin Lee¹, Ali Payani², Duen Horng, Chau

Aff.: ¹Georgia Institute of Technology; ²Cisco Systems Inc.

Arxiv: https://arxiv.org/abs/2410.22312v1

Refer to caption

Method	Annotation	Waterbirds		Biased CelebA		Backgrounds
Method	Annotation	WGA $\uparrow$	MGA $\uparrow$	WGA $\uparrow$	MGA $\uparrow$	MR $\uparrow$	BG-Gap $\downarrow$
Original	-	28.35	72.08	32.60	73.71	78.27	12.99
\hdashline[1pt/1pt] Crayon-Attention	Yes-No	72.31 $\pm$ 0.89	85.23 $\pm$ 0.17	83.29 $\pm$ 2.07	89.61 $\pm$ 0.10	80.85 $\pm$ 0.23	8.52 $\pm$ 0.29
Crayon-Pruning	Yes-No	68.97 $\pm$ 0.43	83.13 $\pm$ 0.08	69.51 $\pm$ 2.20	86.75 $\pm$ 0.40	78.61 $\pm$ 0.15	12.18 $\pm$ 0.11
Crayon-Pruning+Attention	Yes-No	76.04 $\pm$ 3.38	86.03 $\pm$ 0.45	84.38 $\pm$ 1.49	90.13 $\pm$ 0.36	81.66 $\pm$ 0.14	8.40 $\pm$ 0.31
\hdashline[1pt/1pt] JtT [22]	-	46.88 $\pm$ 1.69	78.29 $\pm$ 0.92	35.25 $\pm$ 1.90	74.29 $\pm$ 0.31	77.61 $\pm$ 0.20	12.99 $\pm$ 0.09
MaskTune [32]	-	45.67 $\pm$ 2.17	79.13 $\pm$ 0.36	37.72 $\pm$ 1.37	78.85 $\pm$ 0.28	78.14 $\pm$ 0.08	12.54 $\pm$ 0.07
LfF [13]	-	44.64 $\pm$ 0.85	77.24 $\pm$ 0.27	44.35 $\pm$ 1.76	77.05 $\pm$ 0.40	78.23 $\pm$ 0.05	12.41 $\pm$ 0.08
SoftCon [57]	-	46.10 $\pm$ 4.56	79.93 $\pm$ 1.28	42.38 $\pm$ 13.99	76.17 $\pm$ 3.23	73.14 $\pm$ 2.02	11.58 $\pm$ 0.75
FLAC [17]	-	40.68 $\pm$ 10.58	80.77 $\pm$ 3.59	39.31 $\pm$ 11.84	76.73 $\pm$ 3.51	79.91 $\pm$ 0.36	9.75 $\pm$ 0.35
LC [16]	-	61.65 $\pm$ 3.70	80.43 $\pm$ 0.67	64.98 $\pm$ 5.09	84.80 $\pm$ 0.88	74.78 $\pm$ 1.81	13.35 $\pm$ 0.55
CnC [33]	-	46.98 $\pm$ 1.12	77.80 $\pm$ 0.47	37.76 $\pm$ 0.65	75.34 $\pm$ 0.19	77.87 $\pm$ 0.42	12.91 $\pm$ 0.27
\hdashline[1pt/1pt] RRR [18]	Map	53.96 $\pm$ 3.46	82.29 $\pm$ 0.83	42.16 $\pm$ 4.22	78.64 $\pm$ 0.82	80.08 $\pm$ 0.25	10.68 $\pm$ 0.22
GradMask [58]	Map	58.38 $\pm$ 3.83	82.78 $\pm$ 0.66	8.43 $\pm$ 6.70	65.72 $\pm$ 2.45	80.11 $\pm$ 0.30	10.63 $\pm$ 0.30
ActDiff [59]	Map	64.58 $\pm$ 1.66	84.54 $\pm$ 0.23	41.29 $\pm$ 4.69	79.08 $\pm$ 1.20	75.84 $\pm$ 5.97	11.73 $\pm$ 3.06
GradIA [19]	Yes-No, Map	60.87 $\pm$ 2.86	83.17 $\pm$ 0.51	41.80 $\pm$ 7.82	76.58 $\pm$ 2.09	79.79 $\pm$ 0.19	11.26 $\pm$ 0.15
Bounding Box [44]	Bounding Box	66.36 $\pm$ 2.17	85.85 $\pm$ 0.46	33.34 $\pm$ 1.96	73.03 $\pm$ 0.38	80.82 $\pm$ 0.13	9.81 $\pm$ 0.96
\hdashline[1pt/1pt] ERM	-	22.49 $\pm$ 12.00	70.23 $\pm$ 5.10	37.82 $\pm$ 9.54	71.86 $\pm$ 3.46	77.86 $\pm$ 0.26	12.88 $\pm$ 0.21