notesum.ai

Published at May 11

Instruction Tuning With Loss Over Instructions

NeurIPS

Released Date: May 11, 2024

Authors: Zhengyan Shi¹, Adam X. Yang², Bin Wu¹, Laurence Aitchison², Emine Yilmaz¹, Aldo Lipani¹

Aff.: ¹University College London; ²University of Bristol

Arxiv: https://openreview.net/pdf/2489a69de5d25fdd19c9250d4ef90033722cebc1.pdf

	NLP Benchmarks							LLM-based Evaluation
Method	Understanding & Knowledge	Multi- linguality	Commonsense Reasoning	Math&Code Reasoning	BBH	Safety & Helpfulness	Mean	MT-Bench	AlpacaEval 1.0	AlpacaEval 2.0
Llama-2-Base	63.91	61.99	75.86	13.32	38.80	42.03	49.32	1.16	0.01	0.01
Llama-2-Chat	63.42	55.15	70.28	15.33	38.92	51.79	49.15	6.63	79.04	6.48
Alpagasus Alpaca 5k (5,305 training examples)
\hdashline IT	64.98	57.24	66.06	8.93	26.80	47.74	45.29	3.62	16.29	2.46
Neftune	65.18	56.88	66.45	10.24	29.53	45.46	45.62 $\uparrow$ 0.33	3.50 $\downarrow$ 0.12	21.37 $\uparrow$ 5.08	2.37 $\downarrow$ 0.09
IM (ours)	64.01	56.63	72.47	11.58	35.52	44.62	47.47 $\uparrow$ 2.18	3.48 $\downarrow$ 0.14	19.52 $\uparrow$ 3.23	3.29 $\uparrow$ 0.83
Alpagasus Dolly 3k (2,996 training examples)
\hdashline IT	65.81	57.46	67.55	11.96	33.02	43.70	46.58	4.23	13.42	2.00
Neftune	65.90	57.79	67.28	11.64	35.43	44.36	47.07 $\uparrow$ 0.49	4.42 $\uparrow$ 0.19	14.04 $\uparrow$ 0.62	2.03 $\uparrow$ 0.03
IM (ours)	65.66	57.47	73.24	14.57	37.48	45.29	48.95 $\uparrow$ 2.37	4.06 $\downarrow$ 0.17	15.11 $\uparrow$ 1.69	2.44 $\uparrow$ 0.44
Alpagasus Dolly 9k (9,229 training examples)
\hdashline IT	64.10	56.62	69.70	7.96	32.19	42.65	45.54	4.33	21.54	2.28
Neftune	64.20	56.69	69.51	8.99	33.91	42.62	45.99 $\uparrow$ 0.45	4.21 $\downarrow$ 0.12	31.61 $\uparrow$ 10.07	2.84 $\uparrow$ 0.56
IM (ours)	64.67	55.32	74.87	12.50	36.69	43.96	48.00 $\uparrow$ 2.46	4.55 $\uparrow$ 0.22	30.77 $\uparrow$ 9.23	2.67 $\uparrow$ 0.39
Less Tydiqa (13,533 training examples)
\hdashline IT	64.01	56.81	64.77	12.06	36.54	55.09	48.21	4.08	5.12	1.88
Neftune	64.03	55.09	64.02	13.84	36.65	51.21	47.47 $\downarrow$ 0.74	4.19 $\uparrow$ 0.11	8.35 $\uparrow$ 3.23	2.58 $\uparrow$ 0.70
IM (ours)	64.28	56.10	65.70	17.15	34.86	54.09	48.70 $\uparrow$ 0.49	4.36 $\uparrow$ 0.28	10.10 $\uparrow$ 4.98	2.88 $\uparrow$ 1.00
Less MMLU Chat (13,533 training examples)
\hdashline IT	64.74	57.42	62.94	9.53	33.13	55.35	47.18	3.86	4.42	1.20
Neftune	65.21	57.43	63.14	9.45	35.89	55.32	47.74 $\uparrow$ 0.56	4.06 $\uparrow$ 0.20	6.22 $\uparrow$ 1.80	1.06 $\downarrow$ 0.14
IM (ours)	63.95	56.34	64.76	12.52	36.94	52.55	47.84 $\uparrow$ 0.66	4.54 $\uparrow$ 0.68	9.78 $\uparrow$ 5.36	1.93 $\uparrow$ 0.73
Less BBH ICL (13,533 training examples)
\hdashline IT	63.83	62.04	75.92	6.90	38.93	42.07	48.28	4.78	36.20	2.36
Neftune	63.88	58.83	67.97	13.54	38.63	51.33	49.03 $\uparrow$ 0.75	5.05 $\uparrow$ 0.27	39.81 $\uparrow$ 3.61	2.87 $\uparrow$ 0.51
IM (ours)	64.14	56.72	71.12	13.56	39.03	50.34	49.15 $\uparrow$ 0.87	5.03 $\uparrow$ 0.25	44.15 $\uparrow$ 7.95	3.56 $\uparrow$ 1.20
LIMA (1,030 training examples)
\hdashline IT	63.92	58.29	71.96	16.01	39.27	43.29	48.79	4.77	33.06	2.58
10 epoch Neftune	63.66	57.67	73.03	15.95	38.77	43.14	48.70 $\downarrow$ 0.09	4.79 $\uparrow$ 0.02	30.51 $\downarrow$ 2.55	2.43 $\downarrow$ 0.15
IM (ours)	64.49	58.21	75.55	17.06	38.84	43.45	49.60 $\uparrow$ 0.81	4.83 $\uparrow$ 0.06	32.94 $\downarrow$ 0.12	2.47 $\downarrow$ 0.11