notesum.ai

Published at December 6

Adaptive Dropout for Pruning Conformers

cs.CL

eess.AS

Released Date: December 6, 2024

Authors: Yotaro Kubo¹, Xingyu Cai¹, Michiel Bacchiani¹

Aff.: ¹Google DeepMind

Arxiv: http://arxiv.org/pdf/2412.04836v1

Refer to caption

		dev		test
	#Params	clean	other	clean	other
ConformerL	111M	3.2	7.9	3.4	7.8
Proposed	50.1M	3.0	7.7	3.2	7.7
Shallow (L=9)	62.7M	3.2	8.5	3.5	8.4
Shallow (L=7)	50.6M	3.5	9.3	3.9	9.3
Narrow (D=384)	62.7M	3.2	8.2	3.4	8.4
Narrow (D=320)	43.7M	3.7	9.5	3.9	9.3
NarrowFFN	75.5M	3.6	9.1	3.9	9.3
NoMacaron	75.5M	3.4	9.0	3.8	8.9