notesum.ai

Published at October 18

Do LLMs estimate uncertainty well in instruction-following?

cs.RO

cs.AI

Released Date: October 18, 2024

Authors: Juyeon Heo¹, Miao Xiong², Christina Heinze-Deml³, Jaya Narain³

Aff.: ¹University of Cambridge; ²National University of Singapore; ³Apple

Arxiv: https://arxiv.org/abs/2410.14582v1

Refer to caption

Case	Task quality entanglement
	Inst-following	Task quality	Verbalized confidence
Case 1	o	H	$7.53\pm 0.49$
Case 2	o	L	$7.00\pm 0.63$
Case 3	x	H	$7.42\pm 0.51$
Case 4	x	L	$6.64\pm 0.33$