notesum.ai

Published at November 25

Transparent Neighborhood Approximation for Text Classifier Explanation

cs.CL

Released Date: November 25, 2024

Authors: Yi Cai¹, Arthur Zimek², Eirini Ntoutsi³, Gerhard Wunder¹

Aff.: ¹Dept. of Math. and Comp. Science, Freie Universitat Berlin, Berlin, Germany; ²Dept. of Math. and Comp. Science, University of Southern Denmark, Odense, Denmark; ³Research Institute CODE, Universitat der Bundeswehr München, Munich, Germany

Arxiv: http://arxiv.org/abs/2411.16251v1

Dataset & Model	Explainer	Correctness		Completeness	Compactness	Time cost (s) per entry
Dataset & Model	Explainer	$R^{2}$ score	Fidelity	Confidence drop	AOPC
Amazon & BERT	LIME	0.694 ± 0.197	0.867 ± 0.123	0.346 ± 0.302	0.353 ± 0.246	0.160
	XSPELLS	-	1.000 ± 0.000	0.096 ± 0.225	0.082 ± 0.189	11.852
	ABELE	0.869 ± 0.063	0.942 ± 0.030	0.358 ± 0.398	0.316 ± 0.365	364.359
	XPROAX	0.878 ± 0.065	0.937 ± 0.035	0.583 ± 0.369	0.484 ± 0.329	8.972
	XPROB	0.828 ± 0.070	0.929 ± 0.026	0.584 ± 0.345	0.482 ± 0.311	1.019
Amazon & LSTM	LIME	0.880 ± 0.127	0.896 ± 0.110	0.343 ± 0.293	0.331 ± 0.223	0.023
	XSPELLS	-	1.000 ± 0.000	0.067 ± 0.151	0.101 ± 0.163	10.674
	ABELE	0.874 ± 0.059	0.936 ± 0.040	0.281 ± 0.293	0.273 ± 0.249	239.322
	XPROAX	0.912 ± 0.053	0.938 ± 0.035	0.567 ± 0.285	0.437 ± 0.234	4.582
	XPROB	0.906 ± 0.045	0.939 ± 0.025	0.582 ± 0.245	0.477 ± 0.220	0.804
Yelp & BERT	LIME	0.514 ± 0.280	0.882 ± 0.136	0.366 ± 0.467	0.502 ± 0.421	0.151
	XSPELLS	-	1.000 ± 0.000	0.231 ± 0.408	0.182 ± 0.333	11.852
	ABELE	0.872 ± 0.056	0.950 ± 0.227	0.407 ± 0.474	0.328 ± 0.409	369.802
	XPROAX	0.858 ± 0.064	0.970 ± 0.024	0.770 ± 0.401	0.629 ± 0.366	8.871
	XPROB	0.777 ± 0.071	0.943 ± 0.022	0.760 ± 0.405	0.609 ± 0.372	1.016
Yelp & LSTM	LIME	0.757 ± 0.222	0.954 ± 0.101	0.506 ± 0.424	0.594 ± 0.293	0.023
	XSPELLS	-	1.000 ± 0.000	0.288 ± 0.397	0.225 ± 0.299	10.829
	ABELE	0.867 ± 0.063	0.941 ± 0.039	0.448 ± 0.434	0.334 ± 0.346	233.105
	XPROAX	0.864 ± 0.071	0.958 ± 0.041	0.827 ± 0.299	0.623 ± 0.278	4.508
	XPROB	0.833 ± 0.054	0.945 ± 0.021	0.822 ± 0.296	0.658 ± 0.288	0.815