notesum.ai

Published at December 6

Slicing Vision Transformer for Flexible Inference

cs.CV

cs.LG

Released Date: December 6, 2024

Authors: Yitian Zhang¹, Huseyin Coskun, Xu Ma, Huan Wang, Ke Ma, Xi, Chen, Derek Hao Hu, Yun Fu

Aff.: ¹Snap Inc.

Arxiv: http://arxiv.org/pdf/2412.04786v1

Refer to caption

Method	Param	$r=0.25$		$r=0.50$		$r=0.75$		$r=1.00$
Method	Param	Acc1.	GFLOPs	Acc1.	GFLOPs	Acc1.	GFLOPs	Acc1.	GFLOPs
AutoFormer [4]	22M	50.8 $\%$	0.4	65.6 $\%$	1.3	69.5 $\%$	2.7	69.8 $\%$	4.6
US-Net [37]	22M	52.7 $\%$	0.4	60.1 $\%$	1.3	66.9 $\%$	2.7	73.2 $\%$	4.6
Separate Training	43M	45.8 $\%$	0.4	65.1 $\%$	1.3	70.7 $\%$	2.7	75.0 $\%$	4.6
Scala	22M	58.7 $\%$	0.4	68.3 $\%$	1.3	73.3 $\%$	2.7	76.1 $\%$	4.6