notesum.ai

Published at November 29

DELT: A Simple Diversity-driven EarlyLate Training for Dataset Distillation

cs.CV

cs.AI

cs.LG

Released Date: November 29, 2024

Authors: Zhiqiang Shen¹, Ammar Sherif², Zeyuan Yin¹, Shitong Shao²

Aff.: ¹MBZUAI; ²Unknown

Arxiv: http://arxiv.org/pdf/2411.19946v1

Refer to caption

		ResNet-18			ResNet-101			MobileNet-v2
Dataset	IPC	SRe²L [40]	RDED [32]	DELT (Ours)	SRe²L [40]	RDED [32]	DELT (Ours)	RDED [32]	DELT (Ours)
	1	16.6 $\pm$ 0.9	22.9 $\pm$ 0.4	24.0 $\pm$ 0.8	13.7 $\pm$ 0.2	18.7 $\pm$ 0.1	20.4 $\pm$ 1.0	18.1 $\pm$ 0.9	20.2 $\pm$ 0.4
CIFAR-10	10	29.3 $\pm$ 0.5	37.1 $\pm$ 0.3	43.0 $\pm$ 0.9	24.3 $\pm$ 0.6	33.7 $\pm$ 0.3	37.4 $\pm$ 1.2	29.2 $\pm$ 1.1	29.3 $\pm$ 0.3
	50	45.0 $\pm$ 0.7	62.1 $\pm$ 0.1	64.9 $\pm$ 0.9	34.9 $\pm$ 0.1	51.6 $\pm$ 0.4	54.1 $\pm$ 0.8	39.9 $\pm$ 0.5	42.9 $\pm$ 2.2
	1	19.1 $\pm$ 1.1	35.8 $\pm$ 1.0	24.1 $\pm$ 1.8	15.8 $\pm$ 0.6	25.1 $\pm$ 2.7	19.4 $\pm$ 1.7	26.4 $\pm$ 3.4	19.1 $\pm$ 1.0
ImageNette	10	29.4 $\pm$ 3.0	61.4 $\pm$ 0.4	66.0 $\pm$ 1.4	23.4 $\pm$ 0.8	54.0 $\pm$ 0.4	55.4 $\pm$ 6.2	52.7 $\pm$ 6.6	64.7 $\pm$ 1.4
	50	40.9 $\pm$ 0.3	80.4 $\pm$ 0.4	88.2 $\pm$ 1.2	36.5 $\pm$ 0.7	75.0 $\pm$ 1.2	83.3 $\pm$ 1.1	80.0 $\pm$ 0.0	85.7 $\pm$ 0.4
Tiny-ImageNet	1	2.62 $\pm$ 0.1	9.7 $\pm$ 0.4	9.3 $\pm$ 0.5	1.9 $\pm$ 0.1	3.8 $\pm$ 0.1	5.6 $\pm$ 1.0	3.5 $\pm$ 0.1	3.5 $\pm$ 0.5
	10	16.1 $\pm$ 0.2	41.9 $\pm$ 0.2	43.0 $\pm$ 0.1	14.6 $\pm$ 1.1	22.9 $\pm$ 3.3	42.8 $\pm$ 0.9	24.6 $\pm$ 0.1	26.5 $\pm$ 0.5
	50	41.1 $\pm$ 0.4	58.2 $\pm$ 0.1	55.7 $\pm$ 0.5	42.5 $\pm$ 0.2	41.2 $\pm$ 0.4	58.5 $\pm$ 0.3	49.3 $\pm$ 0.2	51.3 $\pm$ 0.5
ImageNet-100	10	9.5 $\pm$ 0.4	36.0 $\pm$ 0.3	28.2 $\pm$ 1.5	6.4 $\pm$ 0.1	33.9 $\pm$ 0.1	22.4 $\pm$ 3.3	23.6 $\pm$ 0.7	15.8 $\pm$ 0.2
	50	27.0 $\pm$ 0.4	61.6 $\pm$ 0.1	67.9 $\pm$ 0.6	25.7 $\pm$ 0.3	66.0 $\pm$ 0.6	70.8 $\pm$ 2.3	51.5 $\pm$ 0.8	55.0 $\pm$ 1.8
	100	-	74.5 $\pm$ 0.4	75.1 $\pm$ 0.2	-	73.5 $\pm$ 0.8	77.6 $\pm$ 1.8	70.8 $\pm$ 1.1	76.7 $\pm$ 0.3
ImageNet-1K	10	21.3 $\pm$ 0.6	42.0 $\pm$ 0.1	45.8 $\pm$ 0.1	30.9 $\pm$ 0.1	48.3 $\pm$ 1.0	48.5 $\pm$ 1.6	32.3 $\pm$ 0.2	35.1 $\pm$ 0.5
	50	46.8 $\pm$ 0.2	56.5 $\pm$ 0.1	59.2 $\pm$ 0.4	60.8 $\pm$ 0.5	61.2 $\pm$ 0.4	66.1 $\pm$ 0.5	52.8 $\pm$ 0.4	56.2 $\pm$ 0.3
	100	52.8 $\pm$ 0.3	59.8 $\pm$ 0.1	62.4 $\pm$ 0.2	62.8 $\pm$ 0.2	-	67.6 $\pm$ 0.3	56.2 $\pm$ 0.1	58.9 $\pm$ 0.3