notesum.ai

Published at November 16

MetaLA: Unified Optimal Linear Approximation to Softmax Attention Map

cs.AI

Released Date: November 16, 2024

Authors: Yuhong Chou1, Man Yao2, Kexin Wang2, Yuqi Pan2, Ruijie Zhu3, Yiran Zhong4, Yu Qiao4, Jibin Wu1, Bo Xu2, Guoqi Li2

Aff.: 1The Hong Kong Polytechnic University; 2Institute of Automation, Chinese Academy of Sciences; 3UC Santa Cruz; 4Shanghai AI Lab

Arxiv: http://arxiv.org/abs/2411.10741v1