notesum.ai

Published at October 21

Small Contributions, Small Networks: Efficient Neural Network Pruning Based on Relative Importance

cs.LG

cs.AI

Released Date: October 21, 2024

Authors: Mostafa Hussien¹, Mahmoud Afifi², Kim Khoa Nguyen¹, Mohamed Cheriet¹

Aff.: ¹ÉTS, University of Quebec, Canada; ²Google

Arxiv: https://arxiv.org/abs/2410.16151v1

Refer to caption

	Pruning ratio per iteration
Configuration	25%	15%	10%	5%
$\alpha$ = 1, $\beta$ = 1e-7, w/o $s$	97.61	97.45	97.91	97.93
$\alpha$ = 1, $\beta$ = 0, w/ $s$	97.62	97.49	97.90	98.0
$\alpha$ = 1, $\beta$ = 1e-5, w/ $s$	96.14	96.63	97.28	97.59
$\alpha$ = 1, $\beta$ = 1e-7, w/ $s$	97.84	97.52	97.92	98.03