notesum.ai

Published at November 12

Learning with Less: Knowledge Distillation from Large Language Models via Unlabeled Data

cs.AI

Released Date: November 12, 2024

Authors: Juanhui Li1, Sreyashi Nag2, Hui Liu2, Xianfeng Tang2, Sheikh Sarwar2, Limeng Cui2, Hansu Gu2, Suhang Wang3, Qi He2, Jiliang Tang1

Aff.: 1Michigan State University; 2Amazon.com; 3Penn State University

Arxiv: http://arxiv.org/abs/2411.08028v1