notesum.ai

Published at December 10

Quantifying the Prediction Uncertainty of Machine Learning Models for Individual Data

cs.LG

cs.IT

math.IT

Released Date: December 10, 2024

Authors: Koby Bibas¹

Aff.: ¹Tel Aviv University

Arxiv: http://arxiv.org/pdf/2412.07520v1

Refer to caption

IND	OOD	Baseline/+pNML	ODIN/+pNML	Gram/+pNML	OECC/+pNML
CIFAR-100	iSUN	69.7 / 96.4	84.5 / 96.7	99.0 / 99.5	99.2 / 99.5
	LSUN (R)	70.8 / 96.6	86.0 / 96.9	99.3 / 99.7	99.4 / 99.6
	LSUN (C)	80.1 / 93.1	91.5 / 93.1	91.4 / 94.5	93.9 / 96.1
	Imagenet (R)	71.6 / 97.4	85.5 / 97.6	99.0 / 99.5	99.0 / 99.5
	Imagenet (C)	76.2 / 95.7	88.8 / 96.0	97.7 / 98.7	98.2 / 99.0
	Uniform	43.3 / 100	83.7 / 100	100 / 100	99.9 / 100
	Gaussian	30.6 / 100	50.6 / 100	100 / 100	100 / 100
	SVHN	82.6 / 96.2	92.5 / 96.2	97.3 / 98.4	97.0 / 97.5
CIFAR-10	iSUN	94.8 / 98.7	98.9 / 98.9	99.8 / 100	99.9 / 100
	LSUN (R)	95.5 / 98.9	99.2 / 99.2	99.9 / 100	99.9 / 100
	LSUN (C)	93.0 / 96.4	95.8 / 96.4	97.5 / 98.7	98.9 / 99.9
	Imagenet (R)	94.1 / 98.8	98.5 / 99.0	99.7 / 99.9	99.8 / 99.9
	Imagenet (C)	93.8 / 97.7	97.6 / 97.9	99.3 / 99.7	99.5 / 99.9
	Uniform	96.6 / 100	100 / 100	100 / 100	100 / 100
	Gaussian	97.6 / 100	100 / 100	100 / 100	100 / 100
	SVHN	89.9 / 98.4	94.6 / 98.7	99.1 / 99.6	99.6 / 100
SVHN	iSUN	94.4 / 98.7	92.8 / 99.1	99.8 / 99.9	100 / 100
	LSUN (R)	94.1 / 98.4	92.5 / 98.9	99.8 / 100	100 / 100
	LSUN (C)	92.9 / 98.0	88.6 / 98.1	98.6 / 99.4	99.8 / 100
	Imagenet (R)	94.8 / 98.6	93.3 / 99.0	99.7 / 99.9	100 / 100
	Imagenet (C)	94.6 / 98.6	92.8 / 98.8	99.4 / 99.8	100 / 100
	Uniform	93.2 / 99.8	91.6 / 100	99.9 / 100	100 / 100
	Gaussian	97.4 / 99.8	98.9 / 99.9	100 / 100	100 / 100
	CIFAR-10	91.8 / 96.7	88.9 / 97.8	95.4 / 97.3	99.5 / 100
	CIFAR-100	91.4 / 96.7	88.2 / 97.8	96.4 / 98.0	99.6 / 100