notesum.ai

Published at November 27

Training Noise Token Pruning

cs.CV

Released Date: November 27, 2024

Authors: Mingxing Rao¹, Bohan Jiang¹, Daniel Moyer¹

Aff.: ¹Vanderbilt University

Arxiv: http://arxiv.org/abs/2411.18092v1

Refer to caption

		Method	Acc.	GFLOPs	TP (imgs/s)
High Token Regime	GFLOPs $\geq 8.0$	Deit-B-Distil. [19]	82.55	17.68	379
		Top-K [15]	81.82	13.18	504
		Zero-TP [21]	81.92	13.10	433
		DynamicViT [17]	80.91	13.34	498
		ToMe [5]	81.86	12.11	508
		TNT (ours)	81.97	13.11	509
	GFLOPs $\geq 2.0$	Deit-S-Distil. [19]	80.49	4.63	1150
		Top-K [15]	79.81	3.44	1496
		Zero-TP [21]	79.66	3.42	983
		DynamicViT [17]	79.45	3.47	1448
		ToMe [5]	80.12	3.45	1281
		TNT (ours)	79.89	3.41	1443
	GFLOPs $\geq 4.0$	ViT/16 [8]	78.70	9.17	644
		Zero-TP [21]	77.07	6.75	688
		ToMe [5]	77.03	6.97	734
		TNT (ours)	77.32	6.73	842
Low Token Regime	GFLOPs $<8.0$	Deit-B-Distil. [19]	82.55	17.68	379
		Top-K [15]	55.98	5.93	1084
		Zero-TP [21]	19.76	6.18	731
		DynamicViT [17]	11.20	5.79	1102
		ToMe [5]	43.37	5.89	995
		TNT (ours)	59.93	5.87	1095
	GFLOPs $<2.0$	Deit-S-Distil. [19]	80.49	4.63	1150
		Top-K [15]	58.20	1.57	3003
		Zero-TP [21]	27.75	1.63	1485
		DynamicViT [17]	24.51	1.52	3037
		ToMe [5]	62.88	1.55	2486
		TNT (ours)	63.82	1.53	2973
	GFLOPs $<4.0$	ViT/16 [8]	78.70	9.17	644
		Zero-TP [21]	0.63	3.26	1179
		ToMe [5]	6.30	3.26	1485
		TNT (ours)	13.42	2.98	1786