notesum.ai

Published at December 5

GEITje 7B Ultra: A Conversational Model for Dutch

cs.CL

Released Date: December 5, 2024

Authors: Bram Vanroy¹

Aff.: ¹KU Leuven, Dutch Language Institute Leuven, Belgium

Arxiv: http://arxiv.org/pdf/2412.04092v1

model_id	conll_nl	dutch_social	scala_nl	squad_nl	wiki_lingua_nl	mmlu_nl	hellaswag_nl	average
gpt-4-1106-preview	66.44	14.22	72.30	57.81	67.13	70.04	88.29	62.32
mistralai/Mistral-7B-v0.1	58.15	7.94	25.41	62.56	64.24	35.49	19.88	39.10
BramVanroy/GEITje-7B-ultra	42.20	12.78	18.23	53.41	68.30	26.92	25.72	35.37
Rijgersberg/GEITje-7B	47.53	4.36	30.67	56.55	67.58	28.12	11.70	35.22
Rijgersberg/GEITje-7B-chat-v2	42.12	11.06	19.71	59.19	65.55	27.71	18.03	34.77
Rijgersberg/GEITje-7B-chat	50.69	8.16	20.45	54.48	66.92	24.89	9.84	33.63
BramVanroy/GEITje-7B-ultra-sft	39.41	7.00	16.10	53.02	67.74	25.80	12.49	31.65