notesum.ai

Published at November 5

cs.CL

cs.AI

cs.LG

I.2.7; I.2.0

Released Date: November 5, 2024

Authors: Razvan-Gabriel Dumitru¹, Paul-Ioan Clotan², Vikas Yadav³, Darius Peteleaza⁴, Mihai Surdeanu¹

Aff.: ¹University of Arizona; ²Università di Bologna; ³ServiceNow AI; ⁴Lucian Blaga University of Sibiu

Model	Technique	Pruned	Piqa	Hellaswag	Winogrande	Arc Easy	Wikitextv2	Average
			Acc. ( $\uparrow$ )	Acc. ( $\uparrow$ )	Acc. ( $\uparrow$ )	Acc. ( $\uparrow$ )	Perplexity ( $\downarrow$ )	Acc. ( $\uparrow$ )
Llama 3-8B	SliceGPT	30%	59.3%	37.2%	56.4%	42.9%	13.37	49.0%
		35%	57.7%	34.1%	54.3%	39.3%	16.58	46.4%
		40%	57.0%	32.4%	51.8%	35.9%	20.69	44.3%
	Dynamic Slicing	30%	60.4%	38.4%	58.0%	42.4%	12.96	49.8%
		35%	58.4%	36.3%	57.2%	39.3%	15.64	47.8%
		40%	58.1%	34.0%	54.4%	36.8%	19.11	45.8%
Mistral-7B	SliceGPT	30%	62.6%	38.0%	59.7%	51.1%	8.87	52.9%
		35%	58.5%	35.9%	57.6%	42.8%	10.80	48.7%
		40%	57.1%	33.6%	54.1%	38.2%	13.33	45.8%
	Dynamic Slicing	30%	63.1%	38.6%	60.2%	51.7%	8.76	53.4%
		35%	58.5%	34.9%	55.7%	45.8%	10.38	48.8%
		40%	57.9%	31.9%	54.1%	40.1%	12.62	46.0%