notesum.ai

Published at December 4

Tight PAC-Bayesian Risk Certificates for Contrastive Learning

stat.ML

cs.LG

Released Date: December 4, 2024

Authors: Anna van Elst¹, Debarghya Ghoshdastidar²

Aff.: ¹LTCI, Télécom Paris, Institut Polytechnique de Paris; ²School of Computation Information and Technology, Technical University of Munich

Arxiv: http://arxiv.org/pdf/2412.03486v1

Risk Certificate		SimCLR Loss					Contrastive 0-1 Risk
		$\tau=1$	$\tau=0.7$	$\tau=0.5$	$\tau=0.2$		$\tau=1$	$\tau=0.7$	$\tau=0.5$	$\tau=0.2$
	Test Loss	4.945	4.640	4.257	2.7076		0.0601	0.0433	0.0324	0.0199
	kl bound (iid)	7.164	7.674	8.246	9.954		0.497	0.488	0.47	0.432
	Catoni’s bound (iid)	7.095	7.556	7.959	9.446		0.469	0.466	0.435	0.417
	Classic bound (iid)	8.475	8.698	8.910	10.166		0.542	0.540	0.530	0.505
	$f$ -divergence [32]	27.03	30.138	33.27	48.472		3.009	3.099	3.139	2.973
	Th. 4 (ours)	5.537	5.491	5.492	6.223		0.367	0.353	0.342	0.329
	Th. 3 (ours)	5.203	5.328	6.269	43.779		0.129	0.117	0.107	0.093
	$\operatorname{KL}/n$	0.0013	0.0014	0.0014	0.0013		–	–	–	–