notesum.ai

Published at November 22

Ex Uno Pluria: Insights on Ensembling in Low Precision Number Systems

Released Date: November 22, 2024

Authors: Giung Nam, Juho Lee¹

Aff.: ¹Kim Jaechul Graduate School of AI, KAIST, Daejeon, South Korea

Arxiv: http://arxiv.org/abs/2411.14860v1

Refer to caption

		ViT-T/16 (6M)			ViT-S/16 (22M)
Method	System	NLL	ERR	AMB	NLL	ERR	AMB
$\medstar$ Pre-trained	FP32	.932	.243	-	.667	.185	-
RTN	INT-6	.948±.001	.247±.001	-	.671±.001	.185±.001	-
	INT-4	1.23±.001	.315±.001	-	.822±.001	.218±.001	-
LPE-BSR	INT-6	.932±.001	.245±.001	.024±.001	.665±.001	.185±.001	.014±.001
	INT-4	1.30±.001	.298±.001	.489±.001	.821±.001	.211±.001	.268±.001