notesum.ai
Published at December 10Reconstructing Deep Neural Networks: Unleashing the Optimization Potential of Natural Gradient Descent
cs.LG
cs.AI
Released Date: December 10, 2024
Authors: Weihua Liu1, Said Boumaraf2, Jianwu Li1, Chaochao Lin1, Xiabi Liu1, Lijuan Niu3, Naoufel Werghi2
Aff.: 1Beijing Institute of Technology, China; 2Khalifa University of Science and Technology, UAE; 3Chinese Academy of Medical Sciences

| Optimizer | MLP | VGG-16 | ResNet-18 |
|---|---|---|---|
| SGD | 54.37% | 92.64% | 93.02% |
| Adam | 53.86% | 92.12% | 92.93% |
| RAdam | 53.92% | 92.16% | 92.91% |
| SNGD | 55.57% | 94.03% | 94.44% |