notesum.ai

Published at December 10

Adapting to Non-Stationary Environments: Multi-Armed Bandit Enhanced Retrieval-Augmented Generation on Knowledge Graphs

cs.AI

cs.CL

Released Date: December 10, 2024

Authors: Xiaqiang Tang¹, Jian Li², Nan Du², Sihong Xie¹

Aff.: ¹The Hong Kong University of Science and Technology (Guangzhou); ²Tencent Hunyuan

Arxiv: http://arxiv.org/pdf/2412.07618v1

Non-stationarity

Method

Test Hit

\uparrow

Test Recall

\uparrow

Test Retrieval Delay

\downarrow

(second per query)

Retriever update

Retrieval Ensemble

83.74 ± 0.58

67.52 ± 0.77

15.00 ± 0.00

Offline MO-MAB

82.25 ± 2.18

66.05 ± 2.56

13.32 ± 1.90

NN Router (Reis et al. 2019)

81.48 ± 0.28

64.90 ± 0.23

14.09 ± 0.42

LLM Router (lla 2024)

82.19 ± 0.47

67.11 ± 0.52

10.36 ± 3.98

Ours

84.80 ± 0.39

72.24 ± 0.71

5.88 ± 0.99

Domain shift

Retrieval Ensemble

67.93 ± 0.61

68.01± 0.34

15.00 ± 0.00

Offline MO-MAB

64.76 ± 4.40

61.32 ± 2.90

7.78 ± 0.44

NN Router (Reis et al. 2019)

69.57 ± 4.52

63.99 ± 3.94

7.65 ± 1.81

LLM Router (lla 2024)

65.75 ± 0.16

59.33 ± 0.25

9.39 ± 0.06

Ours

76.35 ± 0.69

69.47 ± 0.76

11.63 ± 0.28