notesum.ai

Published at November 27

Curriculum Demonstration Selection for In-Context Learning

cs.CL

Released Date: November 27, 2024

Authors: Duc Anh Vu¹, Nguyen Tran Cong Duy¹, Xiaobao Wu¹, Hoang Minh Nhat¹, Du Mingzhe¹, Nguyen Thanh Thong², Anh Tuan Luu¹

Aff.: ¹Nanyang Technological University Singapore; ²National University of Singapore

Arxiv: http://arxiv.org/abs/2411.18126v1

Refer to caption

Model	Pass	Beyond
CodeLlama-7b	28.12 $\pm$ 0.68	21.75 $\pm$ 0.65
+ KATE	36.85 $\pm$ 0.9	28.34 $\pm$ 1.8
+ CDS (ours)	36.98 $\pm$ 1.93	28.1 $\pm$ 2.0
CodeLlama-13b	33.98 $\pm$ 1.03	26.86 $\pm$ 0.9
+ KATE	39.45 $\pm$ 0.01	32.24 $\pm$ 0.27
+ CDS (ours)	41.54 $\pm$ 0.45	33.18 $\pm$ 0.76
deepseek-coder-6.7b	43.36 $\pm$ 0.68	33.95 $\pm$ 0.52
+ KATE	64.19 $\pm$ 0.9	52.24 $\pm$ 0.11
+ CDS (ours)	64.71 $\pm$ 0.9	51.51 $\pm$ 1.38
starcoder2-3b	50.26 $\pm$ 1.13	39.51 $\pm$ 0.67
+ KATE	57.42 $\pm$ 1.35	46.58 $\pm$ 3.28
+ CDS (ours)	58.98 $\pm$ 0.01	47.15 $\pm$ 1.06