notesum.ai

Published at May 9

DPIC: Decoupling Prompt and Intrinsic Characteristics for LLM Generated Text Detection

NeurIPS

Released Date: May 9, 2024

Authors: Xiao Yu¹, Yuang Qi¹, Kejiang Chen¹, Guoqiang Chen¹, Xi Yang¹, PENGYUAN ZHU², Xiuwei Shang¹, Weiming Zhang¹, Nenghai Yu¹

Aff.: ¹University of Science and Technology of China, China; ²Hefei High-dimensional Data Technology, China

Arxiv: https://openreview.net/pdf/9a83f57578d2530d0ec2e5ef7cb874b3d92d68a9.pdf

Refer to caption

Methods	ChatGPT				GPT-4				Claude3
Methods	XSum	Writing	PubMed	Avg.	XSum	Writing	PubMed	Avg.	XSum	Writing	PubMed	Avg.
RoBERTa-base	0.9150	0.7084	0.6188	0.7474	0.6778	0.5068	0.5309	0.5718	0.8944	0.8036	0.3647	0.6876
RoBERTa-large	0.8507	0.5480	0.6731	0.6906	0.6879	0.3822	0.6067	0.5589	0.9027	0.7128	0.3579	0.6578
RADAR	0.9972	0.9593	0.7372	0.8979	0.9931	0.8593	0.8029	0.8851	0.9952	0.9438	0.8029	0.9139
Likelihood	0.9577	0.9739	0.8776	0.9364	0.7982	0.8553	0.8100	0.8212	0.9760	0.9744	0.9240	0.9581
Entropy	0.3305	0.1901	0.2766	0.2657	0.4364	0.3703	0.3296	0.3788	0.4109	0.0836	0.1686	0.2210
LogRank	0.9584	0.9656	0.8680	0.9307	0.7980	0.8289	0.7997	0.8089	0.9783	0.9732	0.9260	0.9592
LRR	0.9164	0.8962	0.7421	0.8516	0.7453	0.7040	0.6810	0.7101	0.9609	0.9598	0.8334	0.9180
DNA-GPT(Neo-2.7)	0.9040	0.9449	0.7598	0.8696	0.7267	0.8164	0.7163	0.7531	0.9071	0.9655	0.5911	0.8212
DNA-GPT(ChatGPT)	0.8396	0.7898	0.6722	0.7672	0.6146	0.6104	0.5745	0.5998	0.8560	0.8767	0.6729	0.8019
DNA-GPT(Vicuna-7b)	0.6992	0.6695	0.5639	0.6442	0.5594	0.5628	0.5366	0.5529	0.7241	0.7305	0.6001	0.6849
NPR	0.7845	0.9697	0.5483	0.7675	0.5211	0.8276	0.4976	0.6154	0.9232	0.9696	0.7746	0.8891
DetectGPT	0.4594	0.8008	0.3804	0.5469	0.3408	0.6542	0.3675	0.4542	0.4323	0.6800	0.7559	0.6227
Fast-DetectGPT	0.9907	0.9916	0.9021	0.9615	0.9064	0.9611	0.8498	0.9058	0.9942	0.9783	0.9035	0.9587
DPIC(ChatGPT)	1.0000	0.9821	0.9082	0.9634	0.9996	0.9768	0.9438	0.9734	1.0000	0.9950	0.9686	0.9878
DPIC(Vicuna-7b)	0.9976	0.9708	0.8990	0.9558	0.9986	0.9644	0.9394	0.9674	0.9992	0.9943	0.9690	0.9875