notesum.ai

Published at December 3

Improved Localized Machine Unlearning Through the Lens of Memorization

cs.LG

Released Date: December 3, 2024

Authors: Reihaneh Torkzadehmahani¹, Reza Nasirigerdeh, Georgios Kaissis, Daniel Rueckert, Gintare Karolina Dziugaite, Eleni Triantafillou

Aff.: ¹Technical University of Munich

Arxiv: http://arxiv.org/pdf/2412.02432v1

Refer to caption

		Non-IID Forget Set			IID Forget Set
		$\Delta_{\text{forget}}$	$\Delta_{\text{MIA}}$	$\Delta_{\text{test}}$	$\Delta_{\text{forget}}$	$\Delta_{\text{MIA}}$	$\Delta_{\text{test}}$
Full-parameter Unlearning	Retraining(Oracle)	$0.00_{\pm 0.00}$	$0.00_{\pm 0.00}$	$0.00_{\pm 0.00}$	$0.00_{\pm 0.00}$	$0.00_{\pm 0.00}$	$0.00_{\pm 0.00}$
	Fine-tuning	$-5.60_{\pm 0.89}$	$6.07_{\pm 1.10}$	$-1.61_{\pm 0.34}$	$-1.98_{\pm 1.10}$	$1.96_{\pm 1.11}$	$-2.00_{\pm 0.72}$
	NegGrad+	$-4.44_{\pm 0.95}$	$4.92_{\pm 1.14}$	$4.61_{\pm 0.22}$	$1.87_{\pm 1.31}$	$1.89_{\pm 1.30}$	$4.21_{\pm 0.67}$
	NegGrad	$-3.30_{\pm 0.72}$	$3.76_{\pm 0.94}$	$4.60_{\pm 0.18}$	$-15.04_{\pm 0.40}$	$15.39_{\pm 0.39}$	$-1.11_{\pm 0.51}$
	Random Label	$-1.64_{\pm 0.98}$	$2.07_{\pm 1.15}$	$4.33_{\pm 0.19}$	$1.69_{\pm 0.46}$	$1.69_{\pm 0.47}$	$4.93_{\pm 0.47}$
	L1-sparse	$-1.50_{\pm 0.82}$	$-1.01_{\pm 1.03}$	$2.07_{\pm 0.51}$	$1.80_{\pm 1.20}$	$-1.80_{\pm 1.08}$	$0.62_{\pm 0.67}$
	IU³³3The code is adapted from https://github.com/OPTML-Group/Unlearn-Saliency/	$-5.00_{\pm 0.88}$	$5.04_{\pm 0.91}$	$4.18_{\pm 0.19}$	$-2.20_{\pm 0.39}$	$2.19_{\pm 0.38}$	$10.94_{\pm 0.43}$
Localized Unlearning	SSD	$-11.16_{\pm 6.28}$	$11.18_{\pm 6.29}$	$2.68_{\pm 1.18}$	$1.60_{\pm 1.99}$	$1.59_{\pm 1.98}$	$11.58_{\pm 1.03}$
	CritMem-RL ( $\alpha=16\%$ )	$-1.82_{\pm 1.19}$	$1.87_{\pm 1.21}$	$4.86_{\pm 0.19}$	$-2.03_{\pm 0.45}$	$2.05_{\pm 0.45}$	$4.36_{\pm 0.37}$
	Shallowest-RL ( $\alpha=25\%$ )	$1.29_{\pm 1.62}$	$-0.80_{\pm 1.74}$	$5.88_{\pm 0.18}$	$3.41_{\pm 0.77}$	$-3.43_{\pm 0.78}$	$6.43_{\pm 0.52}$
	SalLoc-RL ( $\alpha=30\%$ )	$-2.8_{\pm 1.45}$	$3.30_{\pm 1.54}$	$4.63_{\pm 0.27}$	$-3.81_{\pm 0.40}$	$3.80_{\pm 0.39}$	$4.29_{\pm 0.45}$
	DEL ( $\alpha=30\%$ )	$0.43_{\pm 1.06}$	$0.64_{\pm 1.23}$	$2.23_{\pm 0.25}$	$0.97_{\pm 0.42}$	$-0.97_{\pm 0.40}$	$1.87_{\pm 0.49}$