notesum.ai

Published at November 29

CAdam: Confidence-Based Optimization for Online Learning

cs.LG

cs.AI

stat.ML

Released Date: November 29, 2024

Authors: Shaowen Wang¹, Anan Liu², Jian Xiao², Huan Liu², Yuekui Yang³, Cong Xu⁴, Qianqian Pu², Suncong Zheng², Wei Zhang⁴, Jian Li¹

Aff.: ¹Institute for Interdisciplinary Information Sciences (IIIS), Tsinghua University; ²Machine Learning Platform Department, Tencent TEG; ³Department of Computer Science and Technology, Tsinghua University; ⁴East China Normal University

Arxiv: http://arxiv.org/pdf/2411.19647v1

	DeepFM	WideDeep	DNN	PNN	DCN
Adam	$80.51_{\pm.008}$	$80.47_{\pm.006}$	$80.48_{\pm.014}$	$80.66_{\pm.006}$	$80.51_{\pm.010}$
CAdam	$\mathbf{80.81_{\pm.007}}$	$\mathbf{80.79_{\pm.006}}$	$\mathbf{80.78_{\pm.005}}$	$\mathbf{80.96_{\pm.026}}$	$\mathbf{80.77_{\pm.007}}$
Adam Drop	$-0.36_{\pm.014}$	$-0.43_{\pm.007}$	$-0.41_{\pm.016}$	$-0.23_{\pm.012}$	$-0.54_{\pm.013}$
CAdam Drop	$\mathbf{-0.08_{\pm.014}}$	$\mathbf{-0.14_{\pm.009}}$	$\mathbf{-0.12_{\pm.004}}$	$\mathbf{+0.04_{\pm.031}}$	$\mathbf{-0.28_{\pm.015}}$