notesum.ai

Published at December 9

cs.CV

Released Date: December 9, 2024

Authors: Dongchen Han¹, Yifan Pu, Zhuofan Xia, Yizeng Han, Xuran Pan, Xiu Li, Jiwen Lu, Shiji Song, Gao Huang

Aff.: ¹Tsinghua University

Mask Out Position	None	Loc. $3\!\times\!3$	Loc. $5\!\times\!5$	Loc. $7\!\times\!7$	Rand $9$	Rand $25$	Rand $49$
Softmax Attn	72.2	51.6	24.3	9.0	71.7	71.5	71.1
InLine Attn	70.0	58.0	40.0	20.0	70.0	69.9	69.5