notesum.ai

Published at November 11

AssistRAG: Boosting the Potential of Large Language Models with an Intelligent Information Assistant

cs.CL

cs.AI

cs.IR

Released Date: November 11, 2024

Authors: Yujia Zhou¹, Zheng Liu², Zhicheng Dou³

Aff.: ¹Tsinghua University; ²The Hong Kong Polytechnic University; ³Renmin University of China

Arxiv: http://arxiv.org/abs/2411.06805v1

Refer to caption

Method	Main LLM	HotpotQA			2Wiki			Bamboogle
Method		EM	F1	Prec.	EM	F1	Prec.	EM	F1	Prec.
Baselines without retrieval
CloseBook	$\text{LLaMA2-chat}_{\text{ 7B}}$	13.2	18.4	17.8	14.4	18.2	17.8	10.4	16.3	16.7
CloseBook	$\text{ChatGLM}_{\text{ 6B}}$	15.6	20.4	19.9	15.8	19.5	20.0	12.6	17.6	16.9
CloseBook	$\text{ChatGPT}_{3.5}$	20.0	25.8	26.4	21.6	25.7	24.5	14.4	22.0	22.3
Baselines with retrieval
Naive RAG	$\text{LLaMA2-chat}_{\text{ 7B}}$	18.2	23.0	22.5	17.4	23.7	22.8	15.2	20.4	20.3
Naive RAG	$\text{ChatGLM}_{\text{ 6B}}$	21.8	27.2	25.8	17.8	25.0	25.2	15.8	21.1	20.8
Naive RAG	$\text{ChatGPT}_{3.5}$	24.6	33.0	34.5	23.8	30.2	31.1	18.4	24.4	24.7
ReAct	$\text{ChatGPT}_{3.5}$	26.8	41.7	42.6	25.0	33.0	31.6	28.8	37.7	38.2
IRCoT	$\text{ChatGPT}_{3.5}$	31.4	40.3	41.6	30.8	42.6	42.3	30.2	38.8	37.9
Self-Ask	$\text{ChatGPT}_{3.5}$	28.2	43.1	44.8	28.6	37.5	42.8	23.2	32.8	30.8
Self-RAG	$\textsc{Self-RAG}_{\text{ 7B}}$	31.0	42.4	42.3	35.0	40.7	41.0	29.8	35.5	37.8
LLMLingua	$\text{ChatGPT}_{3.5}$	28.2	40.2	40.0	29.4	38.6	37.8	25.2	31.3	30.8
\hdashlineAssistRAG	$\text{LLaMA2-chat}_{\text{ 7B}}$	32.4	41.5	42.6	36.2	41.0	40.5	33.0	39.6	38.7
AssistRAG	$\text{ChatGLM}_{\text{ 6B}}$	33.0	42.4	43.5	38.0	43.2	42.8	32.8	39.8	39.0
AssistRAG	$\text{ChatGPT}_{3.5}$	34.4	44.8	46.5	39.6	45.6	45.7	34.6	41.4	41.1