notesum.ai

Published at May 7

Unchosen Experts Can Contribute Too: Unleashing MoE Models’ Power by Self-Contrast

NeurIPS

Released Date: May 7, 2024

Authors: Chufan Shi1, Cheng Yang1, Xinyu Zhu2, Jiahao Wang3, Taiqiang Wu3, Siheng Li1, Deng Cai4, Yujiu Yang1, Yu Meng2

Aff.: 1Tsinghua University; 2University of Virginia; 3The University of Hong Kong; 4Tencent AI Lab

Arxiv: https://openreview.net/pdf/067fc4537caa9562aed142113b799e22fd5a39a7.pdf