notesum.ai

Published at October 30

Improving Uncertainty Quantification in Large Language Models via Semantic Embeddings

cs.LG

cs.AI

cs.CL

Released Date: October 30, 2024

Authors: Yashvir S. Grewal¹, Edwin V. Bonilla², Thang D. Bui³

Aff.: ¹Australian National University, Australia; Data61, CSIRO, Australia; ²Data61, CSIRO, Australia; ³Australian National University, Australia

Arxiv: http://arxiv.org/abs/2410.22685v1

Refer to caption

Model	SEU (Ours)		SE
Model	FPR [ $\downarrow$ ]	TPR [ $\uparrow$ ]	FPR [ $\downarrow$ ]	TPR [ $\uparrow$ ]
Llama 3.1 8B	0.2608	0.7767	0.1655	0.6543
Phi 3.5	0.2198	0.7571	0.1738	0.6674
Mistral 7B	0.3293	0.7353	0.2256	0.5568