notesum.ai

Published at November 12

cs.CL

cs.AI

cs.LG

Released Date: November 12, 2024

Authors: Nicholas Kluge Corrêa¹, Aniket Sen¹, Sophia Falk¹, Shiza Fatimah¹

Aff.: ¹Friedrich Wilhelm University of Bonn

Benchmark	$r_{\text{160m}}$	$r_{\text{630m}}$	$r_{\text{1b1}}$	$r_{\text{2b4}}$
ENEM	0.10	-0.45	0.41	0.48
BLUEX	-0.06	0.21	0.52	0.00
OAB Exams	0.34	0.21	0.28	0.24
ASSIN2 RTE	-0.30	0.78	0.74	0.34
ASSIN2 STS	0.58	0.22	0.81	-0.50
FAQUAD NLI	-0.31	0.00	0.00	0.17
HateBR	-0.56	0.65	0.48	-0.36
PT Hate Speech	-0.75	-0.15	0.31	0.49
TweetSentBR	0.17	0.50	0.33	0.88
CALAME-PT	0.92	0.96	0.94	0.86
ARC-Challenge	0.76	0.69	0.82	0.61
HellaSwag	0.93	0.94	0.90	0.89
TruthfulQA	-0.16	0.66	-0.05	-0.30
LAMBADA	0.89	0.90	0.91	0.86