notesum.ai

Published at November 18

Reviving Dormant Memories: Investigating Catastrophic Forgetting in Language Models through Rationale-Guidance Difficulty

cs.AI

cs.CL

Released Date: November 18, 2024

Authors: Huashan Sun¹, Yang Gao¹

Aff.: ¹School of Computer Science and Technology, Beijing Institute of Technology

Arxiv: http://arxiv.org/abs/2411.11932v1

Refer to caption

Method	Allocate	Select	FAP $\uparrow$	F.Ra $\downarrow$	BWT $\uparrow$	FWT $\uparrow$	CAP $\uparrow$
Qwen2-0.5B
Single	-	-	72.64	-	-	-	72.64
Multi	-	-	76.32	-	-	-	76.32
CL	-	-	51.75	23.13	-22.98	0.33	72.97
\hdashlineEA	equal	random	71.78	3.40	-2.02	1.0	73.65
InsCL	instDiff	random	74.06	1.55	0.15	1.27	73.91
RGD	mean	random	74.69	0.59	0.67	1.43	74.07
Llama2-7B
Single	-	-	77.62	-	-	-	77.62
Multi	-	-	80.57	-	-	-	80.57
CL	-	-	66.50	14.91	-14.79	2.54	80.16
\hdashlineEA	equal	random	78.59	2.62	-1.77	2.60	80.22
InsCL	instDiff	random	80.56	0.90	0.94	2.06	79.69
RGD	mean	random	81.07	0.80	1.21	2.33	79.95
Mistral-7B
Single	-	-	79.01	-	-	-	79.01
Multi	-	-	78.79	-	-	-	78.79
CL	-	-	69.09	10.73	-10.27	-0.33	78.68
\hdashlineEA	equal	random	76.01	3.58	-2.88	-0.31	78.69
InsCL	instDiff	random	76.25	3.54	-1.86	-1.02	77.97
RGD	mean	random	76.42	3.45	-1.85	-0.86	78.14
Llama2-13B
Single	-	-	73.91	-	-	-	73.91
Multi	-	-	83.48	-	-	-	83.48
CL	-	-	70.27	13.02	-12.90	3.01	82.17
\hdashlineEA	equal	random	80.81	1.75	-1.27	2.83	81.98
InsCL	instDiff	random	81.86	1.09	-0.07	2.77	81.93
RGD	mean-std	random	81.05	1.75	-1.30	3.10	82.26