notesum.ai

Published at November 25

SAGEval: The frontiers of Satisfactory Agent based NLG Evaluation for reference-free open-ended text

cs.CL

cs.MA

Released Date: November 25, 2024

Authors: Reshmi Ghosh¹, Tianyi Yao¹, Lizzy Chen¹, Sadid Hasan¹, Tianwei Chen¹, Dario Bernal¹, Huitian Jiao¹, H M Sajjad Hossain¹

Aff.: ¹Microsoft

Arxiv: http://arxiv.org/abs/2411.16077v1

$Refer to caption$

Scoring Criteria	Neg	Pos	Total	Definition
Accuracy	41	0	41	2
Semantic
Diversity	31	0	31	12
Cohesion	15	0	15	2
Relevancy	15	0	15	4
Audience
Understandability	9	1	10	11
Audience
Engagement	76	5	81	11
Fairness	5	1	6	7
Sentiment	1	10	11	32