notesum.ai

Published at November 22

Reliable Evaluation of Attribution Maps in CNNs: A Perturbation-Based Approach

cs.CV

cs.AI

Released Date: November 22, 2024

Authors: Lars Nieradzik¹, Henrike Stephani¹, Janis Keuper²

Aff.: ¹Image Processing Department, Fraunhofer ITWM; ²Institute of Machine Learning and Analysis (IMLA), Offenburg University

Arxiv: http://arxiv.org/abs/2411.14946v1

Refer to caption

Attribution Method	Perturb (ours) $\uparrow$	InsBlur $\uparrow$	Del $\downarrow$	Ins $\uparrow$
GradCAM Selvaraju et al. (2019)	0.477 $\pm$ 0.08	0.786 $\pm$ 0.063	0.310 $\pm$ 0.148	0.694 $\pm$ 0.109
GradCAM++ Chattopadhay et al. (2018)	0.496 $\pm$ 0.106	0.772 $\pm$ 0.08	0.321 $\pm$ 0.168	0.679 $\pm$ 0.112
LayerCAM Jiang et al. (2021)	0.494 $\pm$ 0.094	0.766 $\pm$ 0.055	0.337 $\pm$ 0.162	0.666 $\pm$ 0.105
PolyCAMm Englebert et al. (2022)	0.430 $\pm$ 0.098	0.720 $\pm$ 0.064	0.315 $\pm$ 0.206	0.532 $\pm$ 0.029
PolyCAMp Englebert et al. (2022)	0.438 $\pm$ 0.110	0.720 $\pm$ 0.073	0.318 $\pm$ 0.208	0.542 $\pm$ 0.083
PolyCAMpm Englebert et al. (2022)	0.440 $\pm$ 0.102	0.724 $\pm$ 0.064	0.320 $\pm$ 0.203	0.546 $\pm$ 0.077
ReciproCAM Byun and Lee (2023)	0.495 $\pm$ 0.137	0.794 $\pm$ 0.087	0.273 $\pm$ 0.149	0.708 $\pm$ 0.131
ScoreCAM Wang et al. (2020a)	0.492 $\pm$ 0.113	0.764 $\pm$ 0.085	0.337 $\pm$ 0.177	0.661 $\pm$ 0.153
SmoothGradCAM++ Omeiza et al. (2019)	0.441 $\pm$ 0.017	0.708 $\pm$ 0.040	0.427 $\pm$ 0.188	0.598 $\pm$ 0.115
BlurintegratedGradients Xu et al. (2020)	0.454 $\pm$ 0.136	0.758 $\pm$ 0.097	0.266 $\pm$ 0.353	0.521 $\pm$ 0.096
Gradients Simonyan et al. (2014)	0.544 $\pm$ 0.137	0.638 $\pm$ 0.152	0.293 $\pm$ 0.314	0.371 $\pm$ 0.237
GuidedintegratedGradients Kapishnikov et al. (2021)	0.452 $\pm$ 0.138	0.697 $\pm$ 0.116	0.294 $\pm$ 0.319	0.505 $\pm$ 0.121
IntegratedGradients Sundararajan et al. (2017)	0.488 $\pm$ 0.135	0.712 $\pm$ 0.128	0.295 $\pm$ 0.319	0.532 $\pm$ 0.124
SmoothGrad Smilkov et al. (2017)	0.562 $\pm$ 0.148	0.707 $\pm$ 0.128	0.295 $\pm$ 0.319	0.561 $\pm$ 0.140
Canny Canny (1986)	0.315 $\pm$ 0.072	0.564 $\pm$ 0.199	0.332 $\pm$ 0.279	0.401 $\pm$ 0.211
Uniform	0.281 $\pm$ 0.061	0.528 $\pm$ 0.240	0.302 $\pm$ 0.276	0.302 $\pm$ 0.276