notesum.ai

Published at November 25

NormXLogit: The Head-on-Top Never Lies

cs.CL

Released Date: November 25, 2024

Authors: Sina Abbasi¹, Mohammad Reza Modarres¹, Mohammad Taher Pilehvar²

Aff.: ¹Tehran Institute for Advanced Studies, Khatam University, Iran; ²Cardiff University, United Kingdom

Arxiv: http://arxiv.org/abs/2411.16252v1

Refer to caption

	SST-2 (AOPC $\uparrow$ )			MNLI (AOPC $\uparrow$ )			STS-B (Acc $\downarrow$ )
	Llama 2	DeBERTa	BERT	Llama 2	DeBERTa	BERT	Llama 2	DeBERTa	BERT
Random Baseline	0.256	0.266	0.245	0.421	0.445	0.361	0.283	0.430	0.457
Gradient Norm	0.216	0.320	0.331	0.419	0.535	0.460	0.351	0.338	0.374
Gradient×Input	0.236	0.345	0.339	0.442	0.565	0.456	0.255	0.214	0.358
Integrated Gradients	0.220	0.346	0.367	0.448	0.571	0.466	0.237	0.227	0.370
DecompX	N/A	N/A	0.574	N/A	N/A	0.585	N/A	N/A	0.336
$\ell^{2}$ norm	0.299	0.360	0.311	0.420	0.473	0.393	0.251	0.199	0.321
LogAt	0.341	0.377	0.364	0.518	0.548	0.566	0.167	0.423	0.313
NormXLogit	0.341	0.386	0.423	0.519	0.566	0.556	0.233	0.320	0.281