notesum.ai

Published at October 20

cs.AI

cs.CL

cs.HC

cs.RO

Released Date: October 20, 2024

Authors: Weichao Zhou¹, Jiaxin Zhang², Hilaf Hasson², Anu Singh², Wenchao Li¹

Aff.: ¹Boston University; ²Intuit AI Research

Retrieval Model	Embedding Model	HyQE Model	DL19	DL20	COVID	NEWS	Touche
contriever	contriever	-	44.54	42.13	27.32	34.84	16.68
		GPT-4o	53.97	51.93	35.03	41.27	17.78
		GPT-3.5-turbo	53.19	50.04	35.06	42.33	21.02
		Mistral-7b-instruct	52.28	49.62	35.54	42.56	20.78
bge-base-en-v1.5	bge-base-en-v1.5	-	70.39	68.30	69.96	40.94	18.99
		GPT-4o	72.04	69.42	80.29	43.01	19.44
		GPT-3.5-turbo	71.77	68.33	80.13	44.03	20.14
		Mistral-7b-instruct	70.72	69.02	78.93	43.34	21.36
SPLADE++ ED	contriever	-	53.47	53.51	67.35	39.01	20.45
		GPT-4o	60.68	61.66	64.90	44.45	19.17
		GPT-3.5-turbo	60.08	58.27	65.97	44.79	23.01
		Mistral-7b-instruct	57.99	59.59	65.78	44.33	22.32
	bge-base-en-v1.5	-	71.25	68.58	80.45	46.21	21.53
		GPT-4o	72.35	68.96	80.82	46.25	22.11
		GPT-3.5-turbo	71.66	68.83	81.55	46.18	23.15
		Mistral-7b-instruct	71.78	69.06	80.82	45.97	22.80
	E5-large-v2	-	70.18	72.50	76.73	40.65	18.03
		GPT-4o	72.69	71.46	75.87	50.43	20.50
		GPT-3.5-turbo	72.23	71.88	78.29	50.16	23.08
		Mistral-7b-instruct	69.92	72.97	76.90	48.67	22.52
	nomic-embed-text-v1.5	-	66.68	67.28	79.37	45.80	23.93
		GPT-4o	71.45	69.69	78.60	45.94	24.22
		GPT-3.5-turbo	68.87	67.80	80.42	46.05	25.73
		Mistral-7b-instruct	69.20	70.56	78.83	45.93	27.18
	text-embedding-3-large	-	72.52	72.86	83.81	54.14	26.25
		GPT-4o	75.57	72.24	83.40	54.33	25.49
		GPT-3.5-turbo	74.44	72.18	83.59	53.85	27.36
		Mistral-7b-instruct	73.97	72.44	83.30	54.51	26.99