notesum.ai

Published at December 4

Alignment at Pre-training! Towards Native Alignment for Arabic LLMs

cs.CL

Released Date: December 4, 2024

Authors: Juhao Liang¹, Zhenyang Cai, Jianqing Zhu, Huang Huang, Kewei Zong, Bang An, Mosen Alharthi, Juncai He, Lian Zhang, Haizhou Li, Benyou Wang, Jinchao Xu

Aff.: ¹Shenzhen Research Institute of Big Data, Shenzhen, China

Arxiv: http://arxiv.org/pdf/2412.03253v1

Models	ArabicMMLU (koto et al.)	EXAMS	ACVA clean	ACVA all	AraTrust	Avg.
Qwen1.5-7B	46.41	38.34	75.17	75.88	37.16	54.59
Llama3-8B	45.78	46.34	77.49	76.68	54.98	60.25
LLaMA3-Tamed-8B	50.17	46.15	80.17	78.37	55.94	62.14
Jais-30B-v3	44.47	45.78	83.39	79.51	52.30	61.09
Qwen1.5-32B	55.94	52.01	79.99	80.07	49.23	63.45
Qwen1.5-72B	61.23	48.68	82.16	82.24	58.81	66.62
Llama3-70B	65.51	54.78	83.70	80.25	60.54	68.96
LLaMA3-Tamed-70B	66.56	55.49	82.58	81.36	63.41	69.88
ChatGPT 3.5 Turbo	57.70	45.93	74.45	76.88	/	/
GPT-4	72.50	57.76	84.06	79.43	/	/