notesum.ai

Published at November 11

On Active Privacy Auditing in Supervised Fine-tuning for White-Box Language Models

cs.CL

cs.AI

Released Date: November 11, 2024

Authors: Qian Sun¹, Hanpeng Wu¹, Xi Sheryl Zhang²

Aff.: ¹University of Chinese Academy of Sciences, Nanjing; ²Institute of Automation, Chinese Academy of Sciences

Arxiv: http://arxiv.org/abs/2411.07070v2

Refer to caption

Datasets	Balanced accuracy			AUC
Datasets	$A_{loss}$	$A_{black}$	Parsing	$A_{loss}$	$A_{black}$	Parsing
PubMed_RCT	0.719 $\pm$ 0.005	0.668 $\pm$ 0.003	0.741 $\pm$ 0.012	0.745 $\pm$ 0.008	0.751 $\pm$ 0.007	0.775 $\pm$ 0.008
Yelp Reviews	0.704 $\pm$ 0.005	0.643 $\pm$ 0.010	0.723 $\pm$ 0.012	0.722 $\pm$ 0.010	0.736 $\pm$ 0.013	0.755 $\pm$ 0.015
BC5CDR	0.723 $\pm$ 0.004	0.678 $\pm$ 0.014	0.742 $\pm$ 0.004	0.748 $\pm$ 0.013	0.750 $\pm$ 0.011	0.769 $\pm$ 0.010
PubMedQA	0.706 $\pm$ 0.007	0.692 $\pm$ 0.012	0.765 $\pm$ 0.019	0.738 $\pm$ 0.011	0.760 $\pm$ 0.018	0.794 $\pm$ 0.020
Wiki Toxicity	0.702 $\pm$ 0.003	0.664 $\pm$ 0.003	0.717 $\pm$ 0.013	0.737 $\pm$ 0.010	0.742 $\pm$ 0.013	0.758 $\pm$ 0.012
AG News	0.687 $\pm$ 0.003	0.652 $\pm$ 0.013	0.700 $\pm$ 0.013	0.715 $\pm$ 0.013	0.701 $\pm$ 0.013	0.734 $\pm$ 0.012
Sentiment140	0.662 $\pm$ 0.004	0.619 $\pm$ 0.005	0.683 $\pm$ 0.003	0.685 $\pm$ 0.005	0.656 $\pm$ 0.012	0.703 $\pm$ 0.015
CoNLL-2003	0.709 $\pm$ 0.002	0.679 $\pm$ 0.008	0.721 $\pm$ 0.014	0.744 $\pm$ 0.009	0.757 $\pm$ 0.014	0.777 $\pm$ 0.013