notesum.ai

Published at December 4

Fine-Grained Behavior Simulation with Role-Playing Large Language Model on Social Media

cs.CL

cs.AI

cs.CY

Released Date: December 4, 2024

Authors: Kun Li¹, Chenwei Dai², Wei Zhou¹, Songlin Hu²

Aff.: ¹School of Cyber Security, University of Chinese Academy of Sciences, China; ²Institute of Information Engineering, Chinese Academy of Sciences, China

Arxiv: http://arxiv.org/pdf/2412.03148v1

	Reddit			Twitter			Zhihu
	Object	Content	Type	Object	Content	Type	Object	Content	Type
Mistral-7b-Instruct
Zero-Shot	25.95±2.0	19.62±1.4	22.92±0.7	19.99±2.4	27.04±6.5	62.92±0.9	21.56±1.2	19.53±1.6	10.38±0.2
Few-Shot	19.99±0.0	14.18±0.0	24.01±0.0	38.80±0.0	22.47±0.0	56.20±0.0	20.32±0.4	23.51±1.2	14.51±0.2
std-CoT-FT	31.56±0.0	19.46±0.1	31.40±0.0	55.76±0.0	55.56±0.0	84.14±0.0	29.30±0.4	16.61±0.8	17.77±0.0
OM-CoT-FT(ours)	34.58±0.2	21.27±0.2	45.12±0.7	64.38±0.0	56.25±0.0	88.12±0.0	33.19±0.7	28.66±0.5	21.54±0.1
Solar-10.7b-Instruct
Zero-Shot	27.31±1.6	25.31±2.3	20.52±0.8	53.40±1.6	27.72±7.6	71.12±1.4	19.48±1.6	15.00±2.1	10.66±0.5
Few-Shot	18.31±0.0	16.44±0.0	21.30±0.0	38.84±0.0	22.74±0.0	56.20±0.0	19.94±0.1	11.26±0.0	12.46±0.2
std-CoT-FT	28.30±0.0	21.20±0.1	45.33±0.0	62.27±0.0	58.86±0.0	74.04±0.0	24.63±0.3	34.21±0.8	16.18±0.4
OM-CoT-FT(ours)	38.16±0.4	23.23±0.0	47.44±0.1	73.53±0.0	49.04±0.0	87.29±0.0	28.45±0.1	36.65±0.4	19.53±0.0