notesum.ai

Published at November 25

Transformers are Deep Optimizers: Provable In-Context Learning for Deep Model Training

Released Date: November 25, 2024

Authors: Weimin Wu¹, Maojiang Su², Jerry Yao-Chieh Hu¹, Zhao Song³, Han Liu⁴

Aff.: ¹Center for Foundation Models and Generative AI, Northwestern University; ²Department of Information and Computing Science, USTC; ³Simons Institute for the Theory of Computing, UC Berkeley; ⁴Department of Statistics and Data Science, Northwestern University

Arxiv: http://arxiv.org/abs/2411.16549v1