notesum.ai

Published at December 5

cs.LG

Released Date: December 5, 2024

Authors: Runsheng Bai¹, Qiang Liu¹, Bo Liu¹

Aff.: ¹Institution of the Author

LLaMA-7B	4 bit			3.x bit			3 bit			2.x bit
	Bit	PPL		Bit	PPL		Bit	PPL		Bit	PPL
		Wiki	C4		Wiki	C4		Wiki	C4		Wiki	C4
FP16	-	5.68	7.08	-	5.68	7.08	-	5.68	7.08	-	5.68	7.08
SqueezeLLM	4	5.79	7.21	3.24	6.13	7.56	3	6.32	7.75	2.23	11.32	15.69
OmniQuant	4	5.86	7.34	3.24	6.15	7.75	3	6.48	8.19	2.25	9.72	12.79
SKIM	4	5.79	7.20	3.2	6.07	7.52	3	6.21	7.68	2.25	8.99	11.00