notesum.ai

Published at October 23

Mechanisms of Symbol Processing for In-Context Learning in Transformer Networks

cs.LG

cs.AI

cs.RO

Released Date: October 23, 2024

Authors: Paul Smolensky^1,2, Roland Fernandez¹, Zhenghao Herbert Zhou³, Mattia Opper⁴, Jianfeng Gao¹

Aff.: ¹Microsoft Research Deep Learning, Redmond WA 98052, USA; ²Johns Hopkins University Cognitive Science, Baltimore MD 21218, USA; ³Yale University Linguistics, New Haven CT 06511; ⁴University of Edinburgh ILCC, Edinburgh EH8 9AB, UK

Arxiv: https://arxiv.org/abs/2410.17498v1