notesum.ai

Published at October 18

Teaching Models to Balance Resisting and Accepting Persuasion

cs.AR

Released Date: October 18, 2024

Authors: Elias Stengel-Eskin¹, Peter Hase¹, Mohit Bansal¹

Aff.: ¹UNC Chapel Hill

Arxiv: https://arxiv.org/abs/2410.14596v1

Refer to caption

model	NQ1	NQ2	Boolq	TruthfulQA	Avg.
Llama-3.1-70B	$75.95_{\pm\text{ 0.29}}$	$56.88_{\pm\text{ 0.42}}$	$71.99_{\pm\text{ 0.60}}$	$38.47_{\pm\text{ 2.32}}$	$60.82_{\pm\text{ 0.82}}$
+ accept	$79.28_{\pm\text{ 9.98}}$	$85.68_{\pm\text{ 7.52}}$	$90.51_{\pm\text{ 4.32}}$	$87.62_{\pm\text{ 5.93}}$	$85.78_{\pm\text{ 2.09}}$
+ resist	$22.45_{\pm\text{ 37.12}}$	$\mathbf{9.16}_{\pm\text{ 14.82}}$	$\mathbf{26.53}_{\pm\text{ 5.54}}$	$\mathbf{2.41}_{\pm\text{ 2.51}}$	$\mathbf{15.13}_{\pm\text{ 13.55}}$
+ PBT	$\mathbf{9.63}_{\pm\text{ 3.74}}$	$16.13_{\pm\text{ 4.10}}$	$37.45_{\pm\text{ 13.71}}$	$27.54_{\pm\text{ 8.13}}$	$22.69_{\pm\text{ 4.02}}$