notesum.ai

Published at November 13

cs.AI

Released Date: November 13, 2024

Authors: Anum Afzal¹, Juraj Vladika¹, Gentrit Fazlija¹, Andrei Staradubets¹, Florian Matthes¹

Aff.: ¹Technical University of Munich, Germany

Model		er	cpr	icl	icl-er	mq-er	mq-cpr	mq-cpr-icl	mq-icl-er
Llama 2 7B	de	8.64	8.64	8.64	7.41	13.58	13.58	13.58	13.58
Llama 2 7B	en	43.21	43.21	43.21	43.21	53.09	53.09	53.09	53.09
Llama 2 13B	de	28.40	28.40	28.40	28.40	34.57	34.57	34.57	34.57
Llama 2 13B	en	50.62	50.62	50.62	50.62	55.56	55.56	55.56	55.56
GPT 3.5	de	56.79	56.79	58.02	55.56	66.67	64.20	70.37	65.43
GPT 3.5	en	44.44	41.98	41.98	43.21	46.91	44.44	45.68	45.68
GPT-4	de	61.73	61.73	61.73	61.73	69.14	66.67	65.43	66.67
GPT-4	en	65.43	66.67	66.67	66.67	75.31	72.84	72.84	72.84
Mistral 7B	de	39.51	39.51	39.51	39.51	48.15	51.85	51.85	49.38
Mistral 7B	en	51.85	51.85	53.09	51.85	56.79	56.79	56.79	56.79