notesum.ai

Published at October 21

Addressing Spectral Bias of Deep Neural Networks by Multi-Grade Deep Learning

cond-mat.soft

cond-mat.dis-nn

cond-mat.mes-hall

cs.AI

cs.CL

Released Date: October 21, 2024

Authors: Ronglong Fang¹, Yuesheng Xu¹

Aff.: ¹Old Dominion University

Arxiv: https://arxiv.org/abs/2410.16105v1

Refer to caption

setting	model	$t_{max}$	$t_{min}$	batch size	time(s)	TrRSE	VaRSE	TeRSE
1	SGDL	$10^{-4}$	$10^{-4}$	$256$	$32,401$	$1.5\times 10^{-1}$	$1.3\times 10^{-1}$	$1.2\times 10^{-1}$
1	MGDL	$10^{-4}$	$10^{-4}$	$256$	$27,817$	$\mathbf{5.9\times 10^{-6}}$	$\mathbf{1.9\times 10^{-5}}$	$\mathbf{1.7\times 10^{-5}}$
2	SGDL	$5\times 10^{-4}$	$10^{-4}$	$256$	$32,053$	$5.8\times 10^{-3}$	$5.9\times 10^{-3}$	$5.7\times 10^{-3}$
2	MGDL	$10^{-4}$	$10^{-4}$	$256$	$27,628$	$\mathbf{1.5\times 10^{-6}}$	$\mathbf{4.0\times 10^{-6}}$	$\mathbf{6.5\times 10^{-6}}$
3	SGDL	$10^{-3}$	$10^{-4}$	$256$	$31,808$	$9.6\times 10^{-2}$	$8.3\times 10^{-2}$	$1.1\times 10^{-1}$
3	MGDL	$10^{-4}$	$10^{-4}$	$256$	$24,876$	$\mathbf{5.2\times 10^{-6}}$	$\mathbf{3.2\times 10^{-5}}$	$\mathbf{2.1\times 10^{-5}}$
4	SGDL	$5\times 10^{-5}$	$10^{-5}$	$256$	$41,063$	$7.9\times 10^{-1}$	$7.5\times 10^{-1}$	$7.7\times 10^{-1}$
4	MGDL	$10^{-4}$	$10^{-4}$	Full	$9,875$	$\mathbf{1.1\times 10^{-3}}$	$\mathbf{2.2\times 10^{-3}}$	$\mathbf{1.3\times 10^{-3}}$