notesum.ai

Published at December 3

What should a neuron aim for? Designing local objective functions based on information theory

cs.IT

cs.LG

cs.NE

math.IT

Released Date: December 3, 2024

Authors: Andreas C. Schneider¹, Valentin Neuhaus², David A. Ehrlich³, Abdullah Makkeh³, Alexander S. Ecker⁴, Viola Priesemann², Michael Wibral³

Aff.: ¹University of Göttingen; ²Max Planck Institute for Dynamics and Self-Organization; ³Campus Institute for Dynamics of Biological Networks, University of Göttingen; ⁴Institute of Computer Science and Campus Institute Data Science, University of Göttingen

Arxiv: http://arxiv.org/pdf/2412.02482v1

Parameter	Backprop Model	Hidden Layer	Output Layer
$N_{\mathrm{Epochs}}$	100	100	100
$N_{\mathrm{Batch}}$	1024	1024*	1024*
Optimizer	Adam	Adam	Adam
Learning rate $\eta$	0.001	0.002	0.003
Weight decay	0.0	0.00035	0.00015
Number of bins per dim.	-	20**	20**
Binning ranges	-	(-20,20)	adaptive
Objective function	cross-entropy	trivariate $\bm{\gamma}$	$\bm{\gamma}=(-0.2,0.1,1.0,0.1,0.0)^{T}$