notesum.ai

Published at November 25

Learning by Analogy: Enhancing Few-Shot Prompting for Math Word Problem Solving with Computational Graph-Based Retrieval

cs.CL

Released Date: November 25, 2024

Authors: Xiaocong Yang¹, Jiacheng Lin¹, Ziqi Wang¹, Chengxiang Zhai¹

Aff.: ¹University of Illinois Urbana-Champaign

Arxiv: http://arxiv.org/abs/2411.16454v1

Refer to caption

	Math23k	ape210k	gsm8k	math_qa	Calc-ape210k	aqua_rat	Avg.
Random_Qwen-0.5B	28.9	19.2	17.1	16.5	12.0	18.1	18.6
BGE_Qwen-0.5B	43.1	39.7	21.2	27.3	17.6	16.9	27.6
Ours_Qwen-0.5B	57.6	49.2	22.7	26.6	30.5	18.9	34.3
Random_{LLaMA-1B/Qwen-1.5B}	50.3	32.7	38.6	17.2	22.8	14.2	27.6
BGE_{LLaMA-1B/Qwen-1.5B}	58.7	50.4	38.7	45.9	20.4	29.9	40.7
Ours_{LLaMA-1B/Qwen-1.5B}	66.6	59.2	40.7	47.3	31.3	37.4	47.1
Random_{LLaMA-3B/Qwen-3B}	68.0	44.3	71.4	52.9	32.6	46.9	52.7
BGE_{LLaMA-3B/Qwen-3B}	73.1	54.6	71.5	64.9	31.5	50.0	57.6
Ours_{LLaMA-3B/Qwen-3B}	78.3	59.9	71.9	64.3	39.8	50.6	60.8
Random_{LLaMA-8B/Qwen-7B}	83.9	62.8	80.1	51.3	30.6	49.6	59.7
BGE_{LLaMA-8B/Qwen-7B}	87.6	73.8	80.4	66.4	39.5	49.6	66.2
Ours_{LLaMA-8B/Qwen-7B}	90.4	76.7	79.2	66.8	46.5	53.1	68.8
Random_{LLaMA-70B/Qwen-72B}	84.7	68.9	84.7	60.6	39.3	59.8	66.3
BGE_{LLaMA-70B/Qwen-72B}	90.9	79.5	86.0	68.5	47.9	64.2	72.8
Ours_{LLaMA-70B/Qwen-72B}	92.4	80.9	87.3	68.0	53.5	64.2	74.4