notesum.ai

Published at October 30

cs.HC

cs.AI

cs.LG

Released Date: October 30, 2024

Authors: Vinay Koshy¹, Frederick Choi¹, Yi-Shyuan Chiang¹, Hari Sundaram¹, Eshwar Chandrasekharan¹, Karrie Karahalios¹

Aff.: ¹University of Illinois, Urbana-Champaign, USA

Model	AUROC	Accuracy	Precision	Recall
Toxicity (Rater-level Annotations)	0.8798	0.8191	0.7102	0.6445
Toxicity (Majority Vote)	0.9196	0.8688	0.7045	0.6780
Disagreement (Rater-Aware)	0.7376	0.7338	0.6373	0.4480
Disagreement (Rater-Blind)	0.6946	0.7004	0.5453	0.4261