notesum.ai

Published at November 20

cs.IR

cs.AI

Released Date: November 20, 2024

Authors: Zhicong Li¹, Jiahao Wang², Zhishu Jiang³, Hangyu Mao⁴, Zhongxia Chen⁴, Jiazhen Du⁴, Yuanxing Zhang⁴, Fuzheng Zhang⁴, Di Zhang⁴, Yong Liu¹

Aff.: ¹Renmin University of China; ²Southeast University; ³Beijing Jiaotong University; ⁴Kuaishou Technology

Method	AmbigNQ				HotpotQA				FreshQA
Method	H@5	P@5	EM	F1	H@5	P@5	EM	F1	H@5	P@5	Acc
OQR	80.04	47.02	51.28	63.47	42.24	18.99	35.81	47.83	71.50	45.57	69.00
Finetuning-based, Single-query rewriting
RRR	73.47	44.78	49.28	62.17	38.39	18.28	36.49	48.05	65.67	40.53	66.50
RQ-RAG	-	-	52.48	63.96	-	-	40.32	54.11	-	-	69.67
Prompt-based, Single-query rewriting with GPT-4
Rewrite	81.06	47.17	51.24	63.96	42.08	18.91	35.47	47.39	70.67	44.61	70.83
Hyde	79.65	59.36	53.95	64.83	46.92	25.23	39.36	51.74	61.10	44.04	62.83
Prompt-based, Multi-query rewriting with GPT-4
RAG-Fusion	86.33	53.62	55.47	68.59	51.58	26.91	40.00	53.56	76.17	60.00	74.50
DMQR-RAG (ours)	88.08	62.43	55.24	68.57	54.18	27.93	41.12	53.99	77.83	60.03	76.67