notesum.ai

Published at October 22

DEAN: Deactivating the Coupled Neurons to Mitigate Fairness-Privacy Conflicts in Large Language Models

cs.CR

cs.AI

cs.DC

cs.LO

Released Date: October 22, 2024

Authors: Chen Qian¹, Dongrui Liu², Jie Zhang³, Yong Liu¹, Jing Shao²

Aff.: ¹Gaoling School of Artificial Intelligence, Renmin University of China, Beijing, China; ²Shanghai Artificial Intelligence Laboratory, Shanghai, China; ³University of Chinese Academy of Sciences, Beijing, China

Arxiv: https://arxiv.org/abs/2410.16672v1

Method	Qwen2-7B-IT		Mistral-7B-IT-v0.2		Vicuna-7B-v1.5		Llama2-7B-Chat
Method	Fairness $\uparrow$	Privacy $\uparrow$	Fairness $\uparrow$	Privacy $\uparrow$	Fairness $\uparrow$	Privacy $\uparrow$	Fairness $\uparrow$	Privacy $\uparrow$
Origin	0.6684	0.7412	0.6231	0.6636	0.5501	0.3760	0.7386	0.7504
FFT	0.5418	0.7900	0.5570	0.7793	0.4046	0.5297	0.5478	0.6758
LoRA	0.4453	0.7656	0.5062	0.7473	0.3857	0.4871	0.5769	0.6164
DoRA	0.4393	0.7793	0.4697	0.7047	0.3783	0.4703	0.5783	0.6195
ReFT	0.3543	0.7991	0.2846	0.5556	0.3626	0.3227	0.3917	0.3577
DEAN	0.7497	0.8447	0.6342	0.7154	0.5778	0.4414	0.7746	0.8432