notesum.ai

Published at December 6

Uncertainty Quantification for Transformer Models for Dark-Pattern Detection

cs.LG

cs.AI

cs.CL

math.PR

Released Date: December 6, 2024

Authors: Javier Muñoz¹, Álvaro Huertas-García, Carlos Martí-González, Enrique De Miguel Ambite

Aff.: ¹Advantx Technological Foundation (Funditec), Madrid, Spain

Arxiv: http://arxiv.org/pdf/2412.05251v1

Refer to caption

		Params	Trainable Params	Accuracy	F1	Inference (ms)	Train emissions (g)	Test emissions (g)
BERT	SNGP	$335142912$	$335142912$	0.957	0.957	30.34 $\pm$ 5.73	1.938	0.031
	DNN	$336192513$	$336192513$	0.955	0.955	27.06 $\pm$ 6.41	1.315	0.03
	BNN	$337243138$	$337243138$	0.968	0.967	279.7 $\pm$ 62.5	1.690	0.312
Mistral	SNGP	$1196184576$	$1024$	0.6165	0.6314	132.38 $\pm$ 8.74	46.045	0.278
	DNN	$1200379905$	$4196353$	0.9407	0.9394	125.42 $\pm$ 10.41	12.977	0.28
	BNN	$1204576258$	$8392706$	0.9407	0.9402	1457.2 $\pm$ 123.1	14.378	2.836
Mamba	SNGP	$371517440$	$371517440$	0.9004	0.9011	59.21 $\pm$ 3.08	16.492	0.062
	DNN	$372567041$	$372567041$	0.9237	0.9237	69.9 $\pm$ 9.5	7.058	0.063
	BNN	$373617666$	$373617666$	0.917	0.917	658.6 $\pm$ 94.2	7.304	0.645
Nomic	SNGP	$136732672$	$136732672$	0.9576	0.9567	18.35 $\pm$ 2.07	0.514	0.014
	DNN	$137520129$	$137520129$	0.9555	0.9552	17.01 $\pm$ 1.56	1.655	0.014
	BNN	$138308610$	$138308610$	0.9640	0.9633	190.8 $\pm$ 30.1	2.135	0.156
Llama	SNGP	$1128829952$	$1024$	0.887	0.890	155.49 $\pm$ 13.68	29.842	0.27
	DNN	$1133025281$	$4196353$	0.921	0.922	149.69 $\pm$ 13.27	11.537	0.266
	BNN	$1137221634$	$8392706$	0.883	0.880	1711.1 $\pm$ 151.3	10.487	2.758