notesum.ai

Published at November 27

Preserving Deep Representations In One-Shot Pruning: A Hessian-Free Second-Order Optimization Framework

cs.LG

Released Date: November 27, 2024

Authors: Ryan Lucas¹, Rahul Mazumder¹

Aff.: ¹MIT

Arxiv: http://arxiv.org/abs/2411.18376v1

Refer to caption

Method	CIFAR-10 (ResNet20)		CIFAR-100 (ResNet50)		ImageNet-1k (ResNet50)
	1:4	2:4	1:4	2:4	1:4	2:4
Dense Model	92.58		77.40		75.71
OBC	76.77	90.98	69.79	77.03	46.29	73.74
+SNOWS	89.09	91.86	76.05	77.11	66.33	73.87
SparseGPT	11.06	86.05	8.27	74.10	1.61	69.46
+SNOWS	86.10	91.94	73.66	77.10	49.32	73.59
MP	10.99	63.45	9.45	70.76	—	63.45
+SNOWS	87.82	91.88	75.87	77.24	62.75	73.97