notesum.ai

Published at November 27

Is my Meeting Summary Good? Estimating Quality with a Multi-LLM Evaluator

cs.CL

cs.AI

Released Date: November 27, 2024

Authors: Frederic Kirstein¹, Terry Ruas¹, Bela Gipp¹

Aff.: ¹University of Göttingen, Germany

Arxiv: http://arxiv.org/abs/2411.18444v1

Refer to caption

Setup	OM	REP	INC	COR	HAL	LAN	STR	IRR
single	4.08	3.74	4.03	3.39	3.81	3.76	3.83	3.38
single	(0.01)	(0.07)	(0.07)	(0.26)	(0.29)	(0.06)	(0.11)	(0.08)
MADP-S	4.30	3.93	4.05	3.96	3.94	3.80	4.03	3.74
MADP-S	(0.03)	(0.00)	(0.04)	(0.11)	(0.23)	(0.07)	(0.01)	(0.04)
MADP-M	4.31	3.95	3.98	3.91	3.98	3.78	4.05	3.76
MADP-M	(0.04)	(0.05)	(0.05)	(0.14)	(0.22)	(0.03)	(0.09)	(0.07)