notesum.ai

Published at December 9

cs.AI

Released Date: December 9, 2024

Authors: Yiding Wang¹, Yuxuan Chen², Fangwei Zhong³, Long Ma¹, Yizhou Wang¹

Aff.: ¹Peking University; ²The University of Hong Kong; ³Beijing Normal University

Human Annotator Agreement	Proportion	GPT-4o Consistency Rate (%)
High consensus ( $>75\%$ agreement)	11/19	100
Moderate preference ( $50.1\%$ – $74.9\%$ )	5/19	80.0
Difficult to distinguish ( $50\%$ agreement)	3/19	66.7