notesum.ai
Published at November 6A Collaborative Content Moderation Framework for Toxicity Detection based on Conformalized Estimates of Annotation Disagreement
cs.CL
cs.AI
cs.HC
68T50 (Primary) 68T37 (Secondary)
I.2.7; I.2.1
Released Date: November 6, 2024
Authors: Guillermo Villate-Castillo, Javier Del Ser, Borja Sanz
| Task | RQ1 | RQ2 | RQ3 |
| Content toxicity detection (primary, classification) | F1 score, Logistic loss, ECE, ACE | Marginal Coverage, MURE, prediction length, certainty F1 score, | R-F1 |
| Annotation ambiguity estimation (auxiliary, regression) | MAE, MSE | Mean interval size, ICP, DI, | R-F1, CARE, |