notesum.ai

Published at May 6

Make Continual Learning Stronger via C-Flat

NeurIPS

Released Date: May 6, 2024

Authors: Ang Bian¹, Wei Li², Hangjie Yuan³, yu chengrong, Mang Wang⁴, Zixiang Zhao⁵, Aojun Lu¹, Pengliang Ji⁶, Tao Feng⁷

Aff.: ¹Sichuan University; ²Sichuan University, Tsinghua University; ³DAMO Academy, Alibaba Group, Zhejiang University; ⁴ByteDance; ⁵Xi'an Jiaotong University; ⁶Carnegie Mellon University; ⁷Tsinghua University

Arxiv: https://openreview.net/pdf/be179393fb5b55da27facef791300b7cea7f22b0.pdf

Level	Speed	Boost (SGD/SAM)
L1	SGD > SAM > C-Flat	+2.39%/+1.91%
L2	SGD > C-Flat > SAM	+1.52%/+1.04%
L3	C-Flat > SGD	+1.51%/+1.03%