notesum.ai

Published at December 4

Global MMLU: Understanding and Addressing Cultural and Linguistic Biases in Multilingual Evaluation

cs.CL

Released Date: December 4, 2024

Authors: Shivalika Singh¹, Angelika Romanou², Clémentine Fourrier³, David I. Adelani⁴, Jian Gang Ngui⁵, Daniel Vila-Suero³, Peerat Limkonchotiwat⁵, Kelly Marchisio⁶, Wei Qi Leong⁵, Yosephine Susanto⁵, Raymond Ng⁵, Shayne Longpre⁷, Wei-Yin Ko⁶, Madeline Smith¹, Antoine Bosselut², Alice Oh⁸, Andre F. T. Martins, Leshem Choshen⁹, Daphne Ippolito¹⁰, Enzo Ferrante³, Marzieh Fadaee¹, Beyza Ermis¹, Sara Hooker¹

Aff.: ¹Cohere For AI; ²EPFL; ³Hugging Face; ⁴Mila, McGill University & Canada CIFAR AI Chair; ⁵AI Singapore, National University of Singapore; ⁶Cohere; ⁷MIT; ⁸KAIST; ⁹MIT, MIT-IBM Watson AI Lab; ¹⁰Carnegie Mellon University, CONICET & Universidad de Buenos Aires

Arxiv: http://arxiv.org/pdf/2412.03304v1