notesum.ai

Published at December 10

RAZOR: Sharpening Knowledge by Cutting Bias with Unsupervised Text Rewriting

cs.CL

cs.LG

Released Date: December 10, 2024

Authors: Shuo Yang¹, Bardh Prenkaj¹, Gjergji Kasneci¹

Aff.: ¹Technical University of Munich

Arxiv: http://arxiv.org/pdf/2412.07675v1

Refer to caption

			FEVER				FEVER-Adversarial
			BERT		RoBERTa		BERT		RoBERTa
			Accuracy	F1	Accuracy	F1	Accuracy	F1	Accuracy	F1
		Original	87.93 $\pm$ 0.51	86.65 $\pm$ 0.78	90.05 $\pm$ 0.48	88.90 $\pm$ 0.69	64.10 $\pm$ 1.21	69.81 $\pm$ 1.45	66.99 $\pm$ 1.30	73.13 $\pm$ 1.82
Rewriting	Strategy	EDA	87.99 $\pm$ 0.30	87.03 $\pm$ 0.50	89.25 $\pm$ 0.32	89.00 $\pm$ 0.66	65.20 $\pm$ 0.99	69.55 $\pm$ 0.86	66.35 $\pm$ 1.54	69.98 $\pm$ 1.97
		CrossAug	90.28 $\pm$ 0.42	89.37 $\pm$ 0.54	91.00 $\pm$ 0.46	91.98 $\pm$ 0.78	63.58 $\pm$ 1.20	66.74 $\pm$ 1.56	67.02 $\pm$ 1.44	72.04 $\pm$ 1.77
		ReW	87.50 $\pm$ 0.88	87.76 $\pm$ 1.19	89.09 $\pm$ 1.21	89.44 $\pm$ 2.01	64.52 $\pm$ 2.21	69.85 $\pm$ 2.30	67.23 $\pm$ 2.03	74.20 $\pm$ 2.68
		PoE	86.25 $\pm$ 1.78	86.98 $\pm$ 1.89	90.20 $\pm$ 1.78	89.79 $\pm$ 1.90	65.20 $\pm$ 2.76	70.45 $\pm$ 2.58	68.00 $\pm$ 2.49	72.12 $\pm$ 2.61
		CLEVER	85.13 $\pm$ 1.20	85.33 $\pm$ 2.03	86.55 $\pm$ 1.72	84.54 $\pm$ 2.33	56.16 $\pm$ 2.54	58.93 $\pm$ 2.27	62.63 $\pm$ 2.96	63.35 $\pm$ 2.35
		RAZOR(with Llama)	87.97 $\pm$ 1.20	87.22 $\pm$ 2.03	90.59 $\pm$ 2.32	89.11 $\pm$ 2.56	66.03 $\pm$ 1.95	69.61 $\pm$ 1.88	68.77 $\pm$ 3.01	73.76 $\pm$ 2.50
		RAZOR (with ChatGPT)	88.73 $\pm$ 2.23	88.39 $\pm$ 2.20	90.45 $\pm$ 1.99	90.22 $\pm$ 2.11	66.66 $\pm$ 1.98	73.78 $\pm$ 1.37	69.21 $\pm$ 1.64	76.08 $\pm$ 2.02