notesum.ai

Published at December 10

cs.CL

cs.IR

Released Date: December 10, 2024

Authors: Philipp Christmann¹, Gerhard Weikum¹

Aff.: ¹Max Planck Institute for Informatics Saarland Informatics Campus, Germany

Method $\downarrow$ / Benchmark $\rightarrow$	CompMix	Crag	TimeQuestions	#Parameters
Gpt-3	$0.502$	$-$	$0.224$	$175{,}000$ M
Gpt-4	$0.528$	$\mathbf{0.633}$	$0.306$	$1{,}760{,}000$ M
Llama3 [33] (8B-Instruct)	$0.431$	$0.385$	$0.178$	$8{,}030$ M
Convinse [3]	$0.407$	$0.298$	$0.423$	$362$ M
UniK-Qa [27]	$0.440$	$0.280$	$0.424$	$223$ M
Explaignn [5]	$0.442$	$0.303$	$0.525$	$328$ M
State-of-the-art	$\mathbf{0.565}$	$-$	$0.571$	$-$
	(Spaghetti [38])		(Un-Faith [15])
Quasar (ours)	${0.564}$	$0.362$	$\mathbf{0.754}$	$8{,}218$ M