notesum.ai

Published at December 9

How Certain are Uncertainty Estimates? Three Novel Earth Observation Datasets for Benchmarking Uncertainty Quantification in Machine Learning

cs.LG

cs.AI

eess.IV

Released Date: December 9, 2024

Authors: Yuanyuan Wang¹, Qian Song¹, Dawood Wasif¹, Muhammad Shahzad², Christoph Koller³, Jonathan Bamber⁴, Xiao Xiang Zhu¹

Aff.: ¹Technical University of Munich; ²National University of Sciences & Technology Pakistan; ³German Aerospace Center; ⁴University of Bristol

Arxiv: http://arxiv.org/pdf/2412.06451v1

Refer to caption

	Kendall and Gal 2017 [33]				Loquercio et al. 2020 [37]
Noise Level	$\hat{B}$		$\hat{\sigma}_{B}$		$\hat{B}$		$\hat{\sigma}_{B}$
	$R^{2}$	$\%$ RMSE	$R^{2}$	$\%$ RMSE	$R^{2}$	$\%$ RMSE	$R^{2}$	$\%$ RMSE
0.01	0.9982	05.76%	-55.2789	999.29%	0.9963	08.27%	-47.8880	931.36%
0.05	0.9967	07.76%	0.0212	133.87%	0.9933	11.11%	-0.0968	141.71%
0.10	0.9893	14.01%	0.9705	23.30%	0.9870	15.43%	0.9776	20.32%
0.15	0.9673	24.49%	0.7801	63.48%	0.9795	19.42%	0.8742	48.02%
0.20	0.9556	28.56%	0.3919	96.41%	0.9489	30.64%	0.4940	87.94%