notesum.ai

Published at October 22

RKadiyala at SemEval-2024 Task 8: Black-Box Word-Level Text Boundary Detection in Partially Machine Generated Texts

cs.LG

cs.AI

Released Date: October 22, 2024

Authors: Ram Mohan Rao Kadiyala¹

Aff.: ¹University of Maryland, College Park

Arxiv: https://arxiv.org/abs/2410.16659v1

Refer to caption

Dataset →	Dev set (Seen Generator)				Test set (Unseen Generator)
Model ↓	MAE		MARE		MAE		MARE
approach →	I	II	I	II	I	II	I	II
DeBERTa	3.217	3.174	0.0190	0.0185	22.031	19.347	0.1013	0.1006
DeBERTa-CRF	2.311	2.192	0.0127	0.0124	20.074	18.538	0.0919	0.0906
SpanBERT	6.593	5.918	0.0234	0.0221	28.406	25.229	0.1283	0.1274
SpanBERT-CRF	4.855	4.519	0.0196	0.0191	24.283	20.97	0.1216	0.1209
Longformer	3.52	2.878	0.0168	0.0162	25.985	21.177	0.1285	0.1103
Longformer-CRF	2.782	2.41	0.0142	0.0139	20.941	18.943	0.0964	0.0959
Longformer.pos	3.296	3.075	0.0177	0.0174	23.219	19.502	0.1029	0.1022
Longformer.pos-CRF	2.613	2.406	0.0137	0.0135	20.223	18.542	0.0911	0.0902
Longformer (baseline)	3.53				21.535