notesum.ai

Published at November 20

Hymba: A Hybrid-head Architecture for Small Language Models

cs.CL
cs.AI

Released Date: November 20, 2024

Authors: Xin Dong, Yonggan Fu, Shizhe Diao, Wonmin Byeon, Zijia Chen, Ameya Sunil Mahabaleshwarkar, Shih-Yang Liu, Matthijs Van Keirsbilck, Min-Hung Chen, Yoshi Suhara, Yingyan Lin, Jan Kautz, Pavlo Molchanov

Arxiv: http://arxiv.org/abs/2411.13676v1