notesum.ai

Published at October 22

Emphasizing Discriminative Features for Dataset Distillation in Complex Scenarios

cs.LG

cs.AI

Released Date: October 22, 2024

Authors: Kai Wang¹, Zekai Li¹, Zhi-Qi Cheng², Samir Khaki³, Ahmad Sajedi³, Ramakrishna Vedantam⁴, Konstantinos N Plataniotis³, Alexander Hauptmann², Yang You⁵

Aff.: ¹National University of Singapore, Carnegie Mellon University; ²Carnegie Mellon University; ³University of Toronto; ⁴Independent Researcher; ⁵National University of Singapore

Arxiv: https://arxiv.org/abs/2410.17193v1

Dataset	IPC	DD					Eval. w/ Knowledge Distillation			Full
Dataset	IPC	Random	MTT	FTD	DATM	EDF	SRe2L	RDED	EDF	Full
ImageNette	1	12.6±1.5	47.7± 0.9	52.2 ± 1.0	52.5±1.0	52.6±0.5	20.8±0.2	28.9±0.1	25.7±0.4	87.8±1.0
	10	44.8±1.3	63.0±1.3	67.7±0.7	68.9±0.8	71.0±0.8	50.6±0.8	59.0±1.0	64.5±0.6
	50	60.4±1.4			75.4±0.9	77.8±0.5	73.8±0.6	83.1±0.6	84.8±0.5
ImageWoof	1	11.4±1.3	28.6±0.8	30.1±1.0	30.4±0.7	30.8±1.0	15.8±0.8	18.0±0.3	19.2±0.2	66.5±1.3
	10	20.2±1.2	35.8±1.8	38.8±1.4	40.5±0.6	41.8±0.2	38.4±0.4	40.1±0.2	42.3±0.3
	50	28.2±0.9			47.1±1.1	48.4±0.5	49.2±0.4	60.8±0.5	61.6±0.8
ImageMeow	1	11.2±1.2	30.7±1.6	33.8±1.5	34.0±0.5	34.5±0.2	22.2±0.6	19.2±0.8	20.8±0.5	65.2±0.8
	10	22.4±0.8	40.4±2.2	43.3±0.6	48.9±1.1	52.6±0.4	27.4±0.5	44.2±0.6	48.4±0.7
	50	38.0±0.5			56.8±0.9	58.2±0.6	35.8±0.7	55.0±0.6	58.2±0.9
ImageYellow	1	14.8±1.0	45.2±0.8	47.7±1.1	48.5±0.4	49.4±0.5	31.8±0.7	30.6±0.2	33.5±0.6	83.2±0.9
	10	41.8±1.1	60.0±1.5	62.8±1.4	65.1±0.7	68.2±0.4	48.2±0.5	59.2±0.5	60.8±0.5
	50	54.6±0.5			70.2±0.8	73.2±0.8	57.6±0.9	75.8±0.7	76.2±0.3
ImageFruit	1	12.4±0.9	26.6±0.8	29.1±0.9	30.9±1.0	32.8±0.6	23.4±0.5	33.8±0.4	29.6±0.4	64.4±0.8
	10	20.0±0.6	40.3±0.5	44.9±1.5	45.5±0.9	46.2±0.6	39.2±0.7	45.4±0.6	48.4±0.8
	50	33.6±0.9			50.2±0.5	53.2±0.5	44.2±0.8	54.8±0.9	56.4±0.6
ImageSquawk	1	13.2±1.1	39.4±1.5	40.5±0.9	41.1±0.6	41.8±0.5	21.2±1.0	33.8±0.6	30.5±0.5	86.4±0.8
	10	29.6±1.5	52.3±1.0	58.4±1.5	61.8±1.3	65.4±0.8	39.2±0.3	59.0±0.5	59.4±0.6
	50	52.8±0.4			71.0±1.2	74.8±1.2	56.8±0.4	77.2±1.2	77.8±0.5
ImageNet100	1	2.4±0.3	-	-	9.8±1.1	11.5±1.0	-	9.4±0.2	8.1±0.6	56.4±0.4
	10	15.2±0.5	-	-	20.9±0.8	21.9±0.8	-	28.2±0.8	32.0±0.5
	50	29.7±0.2	-	-	42.7±0.7	44.2±0.6	-	42.8±0.2	45.6±0.5