notesum.ai

Published at November 6

Mitigating Privacy Risks in LLM Embeddings from Embedding Inversion

cs.CR

cs.AI

cs.CL

Released Date: November 6, 2024

Authors: Tiantian Liu¹, Hongwei Yao¹, Tong Wu¹, Zhan Qin¹, Feng Lin¹, Kui Ren¹, Chun Chen¹

Aff.: ¹State Key Laboratory of Blockchain and Data Security, Zhejiang University

Arxiv: http://arxiv.org/abs/2411.05034v1

Model	Method	SST2			NLI			QR			TS
Model	Method	F1(%)	Recall(%)	BLEU	F1(%)	Recall(%)	BLEU	F1(%)	Recall(%)	BLEU	F1(%)	Recall(%)	BLEU
T5	W/ Attack	93.9	93.3	0.836	96.5	95.0	0.789	98.2	97.9	0.976	95.2	94.7	0.901
	FGSM	14.2	16.1	0.092	25.9	19.4	0.121	36.3	34.4	0.230	39.3	38.6	0.245
	FreeLB	39.8	39.4	0.433	42.2	44.9	0.268	46.4	43.6	0.278	49.0	48.8	0.312
	DPforward	9.35	11.9	0.054	23.2	17.4	0.139	21.7	16.4	0.089	21.5	20.3	0.100
	Sanitization	6.75	11.0	0.030	23.2	16.3	0.095	23.5	21.7	0.103	22.6	22.1	0.092
	Ours	4.75	4.40	0.019	5.35	4.47	0.034	3.57	4.14	0.014	3.56	4.44	0.011
RoBERTa	W/ Attack	93.9	93.2	0.836	82.4	82.7	0.831	98.2	97.9	0.981	95.6	94.8	0.912
	FGSM	17.7	17.8	0.112	36.7	30.1	0.278	36.8	34.8	0.212	37.5	36.1	0.235
	FreeLB	18.5	17.6	0.104	35.7	30.2	0.278	51.3	48.4	0.238	49.8	48.7	0.320
	DPforward	14.3	10.1	0.014	23.7	18.9	0.257	24.4	20.5	0.147	22.1	20.1	0.131
	Sanitization	14.6	13.6	0.083	24.7	19.8	0.125	24.0	22.3	0.137	25.9	24.7	0.109
	Ours	4.45	4.41	0.019	3.15	4.21	0.014	2.98	3.23	0.013	3.21	4.12	0.008
MPNet	W/ Attack	93.9	93.4	0.837	83.2	83.3	0.822	98.8	97.9	0.980	96.1	95.2	0.906
	FGSM	17.4	17.6	0.115	36.4	29.8	0.269	37.0	34.8	0.221	37.8	36.8	0.212
	FreeLB	22.7	21.5	0.145	29.3	24.4	0.178	50.8	49.0	0.304	49.0	48.7	0.328
	DPforward	13.7	10.3	0.015	23.1	18.8	0.138	26.5	24.1	0.167	22.8	20.8	0.167
	Sanitization	9.61	17.1	0.036	17.8	15.0	0.088	23.8	21.8	0.118	24.6	23.4	0.116
	Ours	5.15	4.43	0.012	4.55	4.13	0.011	4.12	4.21	0.009	4.31	5.24	0.010
LLaMA2	W/ Attack	93.9	93.1	0.831	83.3	81.1	0.948	98.5	98.1	0.985	96.9	95.9	0.914
	FGSM	14.2	16.1	0.092	43.2	34.5	0.352	37.9	36.6	0.237	38.8	37.3	0.218
	FreeLB	44.3	43.6	0.446	41.1	34.2	0.351	50.6	49.6	0.289	47.1	46.9	0.283
	DPforward	12.2	13.0	0.058	25.4	21.9	0.115	25.7	24.3	0.121	22.0	22.5	0.108
	Sanitization	11.9	13.3	0.108	24.3	20.4	0.125	23.7	22.9	0.134	25.2	24.9	0.142
	Ours	5.63	4.97	0.014	4.43	3.18	0.009	4.13	3.29	0.010	3.53	4.12	0.011