notesum.ai

Published at October 31

Constraint Back-translation Improves Complex Instruction Following of Large Language Models

cs.CL

cs.AI

Released Date: October 31, 2024

Authors: Yunjia Qi, Hao Peng, Xiaozhi Wang, Bin Xu, Lei Hou, Juanzi Li

Arxiv: http://arxiv.org/abs/2410.24175v1

Refer to caption

Model	Backbone	IFEval					FollowBench (HSR)						AVG
Model	Backbone	[S]P	[S]I	[L]P	[L]I	AVG	L1	L2	L3	L4	L5	AVG	AVG
GPT-3.5*	GPT	$59.0$	$68.5$	$64.0$	$73.6$	$66.3$	$80.3$	$68.0$	$68.6$	$61.1$	$53.2$	$66.2$	$66.3$
GPT-4 $\dagger$	GPT	$76.9$	$83.6$	$79.3$	$85.4$	$81.3$	$84.7$	$76.1$	$71.3$	$74.5$	$62.4$	$73.8$	$77.6$
Vicuna-v1.5-13B $\dagger$	Llama2	$43.1$	$53.6$	$46.6$	$58.0$	$50.3$	$71.2$	$61.3$	$48.3$	$38.0$	$33.1$	$50.4$	$50.4$
WizardLM-v1.2-13B	Llama2	$43.6$	$54.4$	$48.4$	$59.1$	$51.4$	$61.3$	$51.6$	$43.3$	$37.5$	$29.9$	$44.7$	$48.1$
Conifer_SFT-13B $\dagger$	Llama2	$42.9$	$53.0$	$47.5$	$57.4$	$50.2$	$60.5$	$53.6$	$48.4$	$40.7$	$31.7$	$47.0$	$48.6$
Zephyr-beta-7B $\dagger$	Mistral	$32.0$	$46.8$	$44.9$	$58.0$	$45.4$	$57.6$	$51.9$	$41.9$	$41.4$	$31.4$	$44.8$	$45.1$
Conifer_SFT-7B $\dagger$	Mistral	$45.8$	$57.1$	$50.8$	$62.0$	$53.9$	$54.3$	$49.5$	$49.3$	$40.8$	$30.5$	$44.9$	$49.4$
Conifer_DPO-7B $\dagger$	Mistral	$48.1$	$59.1$	$52.3$	$63.3$	$55.7$	$60.3$	$53.6$	$48.0$	$47.1$	$41.0$	$50.0$	$52.9$
Llama3 8B	Llama3	$25.7$	$36.8$	$28.1$	$35.1$	$31.4$	$4.8$	$8.7$	$8.8$	$6.0$	$9.8$	$7.6$	$19.5$
Llama3_Crab	Llama3	$39.4$	$50.2$	$43.8$	$54.2$	$46.9$	$57.5$	$44.9$	$34.9$	$25.2$	$20.0$	$36.5$	$41.7$
Llama3_Crab + DPO	Llama3	$40.3$	$52.0$	$47.7$	$58.9$	$49.7$	$64.6$	$49.0$	$41.6$	$35.8$	$36.8$	$45.5$	$47.6$
Mistral 7B	Mistral	$18.5$	$30.8$	$19.6$	$31.9$	$25.2$	$14.3$	$16.6$	$8.3$	$5.8$	$5.5$	$10.1$	$17.7$
Mistral_Crab	Mistral	$47.9$	$57.3$	$51.6$	$61.2$	$54.5$	$63.9$	$54.4$	$40.1$	$30.4$	$27.9$	$43.3$	$48.9$
Mistral_Crab + DPO	Mistral	$49.7$	$61.5$	$57.7$	$68.5$	$59.3$	$66.1$	$53.6$	$53.4$	$42.4$	$31.7$	$49.4$	$54.4$