notesum.ai

Published at November 13

Confidence-aware Denoised Fine-tuning of Off-the-shelf Models for Certified Robustness

cs.CV

cs.AI

cs.CR

cs.LG

Released Date: November 13, 2024

Authors: Suhyeok Jang¹, Seojin Kim¹, Jinwoo Shin¹, Jongheon Jeong²

Aff.: ¹Korea Advanced Institute of Science & Technology (KAIST); ²Korea University

Arxiv: http://arxiv.org/abs/2411.08933v1

{subcaptionblock}0.95

Category	Method	Off-the-shelf	Certified Accuracy at $\varepsilon$ (%)
Category	Method	Off-the-shelf	0.25	0.50	0.75	1.00	1.25	1.50
RS	PixelDP (Lecuyer et al., 2019)		${\text{\small(71.0)}}_{\text{\large 22.0}}$	${\text{\small(44.0)}}_{\text{\large 2.0}}$	-	-	-	-
	Gaussian (Cohen et al., 2019)		${\text{\small(77.0)}}_{\text{\large 61.0}}$	${\text{\small(66.0)}}_{\text{\large 43.0}}$	${\text{\small(66.0)}}_{\text{\large 32.0}}$	${\text{\small(66.0)}}_{\text{\large 22.0}}$	${\text{\small(47.0)}}_{\text{\large 17.0}}$	${\text{\small(47.0)}}_{\text{\large 14.0}}$
	SmoothAdv (Salman et al., 2019)		${\text{\small(85.0)}}_{\text{\large 73.0}}$	${\text{\small(76.0)}}_{\text{\large 58.0}}$	${\text{\small(75.0)}}_{\text{\large 48.0}}$	${\text{\small(57.0)}}_{\text{\large 38.0}}$	${\text{\small(53.0)}}_{\text{\large 33.0}}$	${\text{\small(53.0)}}_{\text{\large 29.0}}$
	Consistency (Jeong & Shin, 2020)		${\text{\small(77.8)}}_{\text{\large 68.8}}$	${\text{\small(75.8)}}_{\text{\large 58.1}}$	${\text{\small(72.9)}}_{\text{\large 48.5}}$	${\text{\small(52.3)}}_{\text{\large 37.8}}$	${\text{\small(52.3)}}_{\text{\large 33.9}}$	${\text{\small(52.3)}}_{\text{\large 29.9}}$
	MACER (Zhai et al., 2020)		${\text{\small(81.0)}}_{\text{\large 71.0}}$	${\text{\small(81.0)}}_{\text{\large 59.0}}$	${\text{\small(66.0)}}_{\text{\large 46.0}}$	${\text{\small(66.0)}}_{\text{\large 38.0}}$	${\text{\small(66.0)}}_{\text{\large 29.0}}$	${\text{\small(45.0)}}_{\text{\large 25.0}}$
	Boosting (Horváth et al., 2022a)		${\text{\small(83.4)}}_{\text{\large 70.6}}$	${\text{\small(76.8)}}_{\text{\large 60.4}}$	${\text{\small(71.6)}}_{\text{\large 52.4}}$	${\text{\small(52.4)}}_{\text{\large 38.8}}$	${\text{\small(52.4)}}_{\text{\large 34.4}}$	${\text{\small(52.4)}}_{\text{\large{30.4}}}$
	DRT (Yang et al., 2022)		${\text{\small(81.5)}}_{\text{\large 70.4}}$	${\text{\small(72.6)}}_{\text{\large 60.2}}$	${\text{\small(71.9)}}_{\text{\large 50.5}}$	${\text{\small(56.1)}}_{\text{\large 39.8}}$	${\text{\small(56.4)}}_{\text{\large{36.0}}}$	${\text{\small(56.4)}}_{\text{\large{30.4}}}$
	SmoothMix (Jeong et al., 2021)		${\text{\small(77.1)}}_{\text{\large 67.9}}$	${\text{\small(77.1)}}_{\text{\large 57.9}}$	${\text{\small(74.2)}}_{\text{\large 47.7}}$	${\text{\small(61.8)}}_{\text{\large 37.2}}$	${\text{\small(61.8)}}_{\text{\large 31.7}}$	${\text{\small(61.8)}}_{\text{\large 25.7}}$
	ACES (Horváth et al., 2022b)		${\text{\small(77.6)}}_{\text{\large 69.0}}$	${\text{\small(73.4)}}_{\text{\large 57.2}}$	${\text{\small(73.4)}}_{\text{\large 47.0}}$	${\text{\small(57.0)}}_{\text{\large 37.8}}$	${\text{\small(57.0)}}_{\text{\large 32.2}}$	${\text{\small(57.0)}}_{\text{\large 27.8}}$
	CAT-RS (Jeong et al., 2023)		${\text{\small(76.3)}}_{\text{\large 68.1}}$	${\text{\small(76.3)}}_{\text{\large 58.8}}$	${\text{\small(76.3)}}_{\text{\large 48.2}}$	${\text{\small(62.3)}}_{\text{\large 38.5}}$	${\text{\small(62.3)}}_{\text{\large 32.7}}$	${\text{\small(62.3)}}_{\text{\large 27.1}}$
DS	Denoised (Salman et al., 2020)		${\text{\small(72.0)}}_{\text{\large 56.0}}$	${\text{\small(62.0)}}_{\text{\large 41.0}}$	${\text{\small(62.0)}}_{\text{\large 28.0}}$	${\text{\small(44.0)}}_{\text{\large 19.0}}$	${\text{\small(42.0)}}_{\text{\large 16.0}}$	${\text{\small(44.0)}}_{\text{\large 13.0}}$
	Score-based Denoised (Lee, 2021)		${\text{\small\phantom{(00.0)}}}_{\text{\large 60.0}}$	${\text{\small\phantom{(00.0)}}}_{\text{\large 42.0}}$	${\text{\small\phantom{(00.0)}}}_{\text{\large 28.0}}$	${\text{\small\phantom{(00.0)}}}_{\text{\large 19.0}}$	${\text{\small\phantom{(00.0)}}}_{\text{\large 11.0}}$	${\text{\small\phantom{(00.0)}}}_{\text{\large 6.0}}$
	Diffusion Denoised^† (Carlini et al., 2023)		${\text{\small(88.1)}}_{\text{\large 76.7}}$	${\text{\small(88.1)}}_{\text{\large 63.0}}$	${\text{\small(88.1)}}_{\text{\large 45.3}}$	${\text{\small(77.0)}}_{\text{\large 32.1}}$	-	-
	Diffusion Denoised^†¹¹1Further fine-tune the classifier on denoised images from CIFAR-10. (Carlini et al., 2023)		${\text{\small(91.2)}}_{\text{\large 79.3}}$	${\text{\small(91.2)}}_{\text{\large 65.5}}$	${\text{\small(91.2)}}_{\text{\large 48.7}}$	${\text{\small(81.5)}}_{\text{\large 35.5}}$	-	-
	Multi-scale Denoised^† (Jeong & Shin, 2024)		-	${\text{\small(90.3)}}_{\text{\large 61.9}}$	-	${\text{\small(85.1)}}_{\text{\large 32.9}}$	-	${\text{\small(79.6)}}_{\text{\large 16.2}}$
	FT-CADIS (Ours)^†		${\text{\small(88.7)}}_{\text{\large{80.3}}}$	${\text{\small(88.7)}}_{\text{\large{68.4}}}$	${\text{\small(88.7)}}_{\text{\large{54.5}}}$	${\text{\small(74.9)}}_{\text{\large{39.9}}}$	${\text{\small(74.9)}}_{\text{\large 31.6}}$	${\text{\small(74.9)}}_{\text{\large 23.5}}$

{subcaptionblock}0.95

Category	Method	Off-the-shelf	Certified Accuracy at $\varepsilon$ (%)
Category	Method	Off-the-shelf	0.50	1.00	1.50	2.00	2.50
RS	PixelDP (Lecuyer et al., 2019)		${\text{\small(33.0)}}_{\text{\large 16.0}}$	-	-	-	-
	Gaussian (Cohen et al., 2019)		${\text{\small(67.0)}}_{\text{\large 49.0}}$	${\text{\small(57.0)}}_{\text{\large 37.0}}$	${\text{\small(57.0)}}_{\text{\large 29.0}}$	${\text{\small(44.0)}}_{\text{\large 19.0}}$	${\text{\small(44.0)}}_{\text{\large 15.0}}$
	SmoothAdv (Salman et al., 2019)		${\text{\small(65.0)}}_{\text{\large 56.0}}$	${\text{\small(55.0)}}_{\text{\large 45.0}}$	${\text{\small(55.0)}}_{\text{\large 38.0}}$	${\text{\small(42.0)}}_{\text{\large 28.0}}$	${\text{\small(42.0)}}_{\text{\large 26.0}}$
	Consistency (Jeong & Shin, 2020)		${\text{\small(55.0)}}_{\text{\large 50.0}}$	${\text{\small(55.0)}}_{\text{\large 44.0}}$	${\text{\small(55.0)}}_{\text{\large 34.0}}$	${\text{\small(41.0)}}_{\text{\large 24.0}}$	${\text{\small(41.0)}}_{\text{\large 21.0}}$
	MACER (Zhai et al., 2020)		${\text{\small(68.0)}}_{\text{\large 57.0}}$	${\text{\small(64.0)}}_{\text{\large 43.0}}$	${\text{\small(64.0)}}_{\text{\large 31.0}}$	${\text{\small(48.0)}}_{\text{\large 25.0}}$	${\text{\small(48.0)}}_{\text{\large 18.0}}$
	Boosting (Horváth et al., 2022a)		${\text{\small(68.0)}}_{\text{\large 57.0}}$	${\text{\small(57.0)}}_{\text{\large 44.6}}$	${\text{\small(57.0)}}_{\text{\large 38.4}}$	${\text{\small(44.6)}}_{\text{\large 28.6}}$	${\text{\small(38.6)}}_{\text{\large 24.6}}$
	DRT (Yang et al., 2022)		${\text{\small(52.2)}}_{\text{\large 46.8}}$	${\text{\small(49.8)}}_{\text{\large 44.4}}$	${\text{\small(49.8)}}_{\text{\large 39.8}}$	${\text{\small(49.8)}}_{\text{\large 30.4}}$	${\text{\small(49.8)}}_{\text{\large 29.0}}$
	SmoothMix (Jeong et al., 2021)		${\text{\small(55.0)}}_{\text{\large 50.0}}$	${\text{\small(55.0)}}_{\text{\large 43.0}}$	${\text{\small(55.0)}}_{\text{\large 38.0}}$	${\text{\small(40.0)}}_{\text{\large 26.0}}$	${\text{\small(40.0)}}_{\text{\large 24.0}}$
	ACES (Horváth et al., 2022b)		${\text{\small(63.2)}}_{\text{\large 54.0}}$	${\text{\small(55.4)}}_{\text{\large 42.2}}$	${\text{\small(55.0)}}_{\text{\large 35.6}}$	${\text{\small(39.2)}}_{\text{\large 25.6}}$	${\text{\small(50.6)}}_{\text{\large 22.0}}$
	CAT-RS (Jeong et al., 2023)		${\text{\small(44.0)}}_{\text{\large 38.0}}$	${\text{\small(44.0)}}_{\text{\large 35.0}}$	${\text{\small(44.0)}}_{\text{\large 31.0}}$	${\text{\small(44.0)}}_{\text{\large 27.0}}$	${\text{\small(44.0)}}_{\text{\large 24.0}}$
DS	Denoised (Salman et al., 2020)		${\text{\small(60.0)}}_{\text{\large 33.0}}$	${\text{\small(38.0)}}_{\text{\large 14.0}}$	${\text{\small(38.0)}}_{\text{\large 6.0}}$	-	-
	Score-based Denoised (Lee, 2021)		${\text{\small\phantom{(00.0)}}}_{\text{\large 41.0}}$	${\text{\small\phantom{(00.0)}}}_{\text{\large 24.0}}$	${\text{\small\phantom{(00.0)}}}_{\text{\large 11.0}}$	-	-
	Diffusion Denoised^†(Carlini et al., 2023)		${\text{\small(82.8)}}_{\text{\large 71.1}}$	${\text{\small(77.1)}}_{\text{\large 54.3}}$	${\text{\small(77.1)}}_{\text{\large 38.1}}$	${\text{\small(60.0)}}_{\text{\large 29.5}}$	-
	Multi-scale Denoised^† (Jeong & Shin, 2024)		${\text{\small(76.6)}}_{\text{\large 54.6}}$	${\text{\small(76.6)}}_{\text{\large 39.8}}$	${\text{\small(76.6)}}_{\text{\large 23.0}}$	${\text{\small(69.0)}}_{\text{\large 14.6}}$	-
	FT-CADIS (Ours)^†		${\text{\small(81.1)}}_{\text{\large{71.9}}}$	${\text{\small(77.0)}}_{\text{\large{60.1}}}$	${\text{\small(77.0)}}_{\text{\large{45.8}}}$	${\text{\small(66.2)}}_{\text{\large{39.4}}}$	${\text{\small(66.2)}}_{\text{\large{30.7}}}$