notesum.ai

Published at December 5

Foundations of the Theory of Performance-Based Ranking

cs.LG

cs.CV

cs.PF

Released Date: December 5, 2024

Authors: Sébastien Piérard¹, Anaïs Halin¹, Anthony Cioppa¹, Adrien Deliège¹, Marc Van Droogenbroeck¹

Aff.: ¹University of Liège

Arxiv: http://arxiv.org/pdf/2412.04227v1

	without any constraint: all performances					with constraint: positive prior = $0.2$					with constraint: positive prior = $0.5$
score	1^st test	2^nd test	3^rd test	$\tau_{min}$	$\tau_{max}$	1^st test	2^nd test	3^rd test	$\tau_{min}$	$\tau_{max}$	1^st test	2^nd test	3^rd test	$\tau_{min}$	$\tau_{max}$
Accuracy	V	V	V	0.469	1^†	V	V	V	0.157	1^†	V	V	V	0.505	1^†
F-score for $\beta=0.5$	V	V	V	0.079	1^†	V	V	V	0.451	1^†	V	V	V	0.352	1^†
F-score for $\beta=1.0$	V	V	V	0.161	1^†	V	V	V	0.352	1^†	V	V	V	0.194	1^†
F-score for $\beta=2.0$	V	V	V	0.079	1^†	V	V	V	0.194	1^†	V	V	V	0.072	1^†
Negative Predictive Value (NPV)	V	V	V	0.000	1^†	V	V	V	0.503	1^†	V	V	V	0.503	1^†
Positive Predictive Value (PPV)	V	V	V	0.000	1^†	V	V	V	0.503	1^†	V	V	V	0.503	1^†
True Negative Rate (TNR)	V	V	V	0.000	1^†	V	V	V	0.000	1^†	V	V	V	0.000	1^†
True Positive Rate (TPR)	V	V	V	0.000	1^†	V	V	V	0.000	1^†	V	V	V	0.000	1^†
Balanced Accuracy	V	X	X	0.486	0.713	V	V	V	0.504	1^†	V	V	V	0.505	1^†
Cohen’s $\kappa$	X	X	X	0.476	0.697	V	V	V	0.503	1^†	V	V	V	0.505	1^†
Informedness	V	X	X	0.486	0.713	V	V	V	0.504	1^†	V	V	V	0.505	1^†
Positive Likelihood Ratio (PLR)	V	X	X	0.420	0.677	V	V	V	0.491	1^†	V	V	V	0.491	1^†
Probability of True Negative (PTN)	X	V	V	-0.007	0.818	V	V	V	0.000	1^†	V	V	V	0.000	1^†
Probability of True Positive (PTP)	X	V	V	-0.006	0.818	V	V	V	0.000	1^†	V	V	V	0.000	1^†
Chance in Cohen’s $\kappa$	X	X	X	0.194	0.498	X	V	V	-0.157	0.849	V	V	V	0^†	0^†
Error Rate	X	V	V	-1^†	-0.469	X	V	V	-1^†	-0.157	X	V	V	-1^†	-0.505
False Discovery Rate (FDR)	X	V	V	-1^†	0.000	X	V	V	-1^†	-0.503	X	V	V	-1^†	-0.503
False Negative Rate (FNR)	X	V	V	-1^†	0.000	X	V	V	-1^†	0.000	X	V	V	-1^†	0.000
False Omission Rate (FOR)	X	V	V	-1^†	0.000	X	V	V	-1^†	-0.503	X	V	V	-1^†	-0.503
False Positive Rate (FPR)	X	V	V	-1^†	0.000	X	V	V	-1^†	0.000	X	V	V	-1^†	0.000
Geometric mean of TNR and TPR	V	X	X	0.461	0.653	V	X	V	0.503	0.831	V	X	V	0.503	0.830
Markedness	V	X	X	0.486	0.713	V	X	X	0.418	0.887	V	X	X	0.503	0.913
Matthews Correlation Coefficient (MCC)	V	X	X	0.503	0.746	V	X	X	0.458	0.944	V	X	X	0.503	0.963
Negative Likelihood Ratio (NLR)	X	X	X	-0.677	-0.418	X	V	V	-1^†	-0.491	X	V	V	-1^†	-0.491
Odds Ratio (OR)	V	X	X	0.499	0.671	V	X	X	0.503	0.894	V	X	X	0.503	0.892
Rate of positive predictions	X	V	V	-0.469	0.469	X	V	V	-0.849	0.157	X	V	V	-0.504	0.505
Sensitivity Index Estimate ( $d^{\prime}$ )	V	X	X	0.502	0.786	V	X	X	0.503	0.926	V	X	X	0.503	0.924