notesum.ai

Published at December 9

Gated Delta Networks: Improving Mamba2 with Delta Rule

cs.CL

cs.LG

Released Date: December 9, 2024

Authors: Songlin Yang¹, Jan Kautz², Ali Hatamizadeh²

Aff.: ¹MIT CSAIL; ²NVIDIA

Arxiv: http://arxiv.org/pdf/2412.06464v1