notesum.ai

Published at December 3

cs.CL

cs.AI

cs.CY

cs.LG

Released Date: December 3, 2024

Authors: Abhas Kumar¹, Kapil Pathak¹, Rajesh Kavuru¹, Prabhakar Srinivasan¹

Aff.: ¹Synechron, Bangalore, India

Model( $M$ )	$L_{r}$	$Q_{b}$	$B_{M}$ SPICE	$F_{T}$ SPICE	$C_{E}$ (gm)	GPT-4o SPICE
Qwen-VL-2B	8	4-bit		0.35	95.5 (base)	0.16
	16		0.16	0.36	97.8 (+2.4%)
	32			0.35	98.8 (+3.4%)
Qwen-VL-7B	8			0.37	137.8 (+44.3%)
	16		0.14	0.37	137.9 (+44.3%)
	32			0.38	138.2 (+44.7%)
Llama-3.2-11B	8			0.30	222.11 (+132.58%)
	16		0.18	0.29	222.46 (+133.%)
	32			0.31	221.51 (132%)