notesum.ai

Published at November 10

Epistemic Integrity in Large Language Models

cs.CL

cs.AI

cs.HC

Released Date: November 10, 2024

Authors: Bijean Ghafouri¹, Shahrad Mohammadzadeh², James Zhou³, Pratheeksha Nair², Jacob-Junqi Tian⁴, Mayank Goel⁵, Reihaneh Rabbany², Jean-François Godbout⁶, Kellin Pelrine²

Aff.: ¹University of Southern California; ²McGill University; ³UC Berkeley; ⁴Vector Institute; ⁵IIIT Hyderabad; ⁶Université de Montréal

Arxiv: http://arxiv.org/abs/2411.06528v1

Model	Anthropic	Pei	LLama3-8b	GM	CMV
Base Pei	1.91	0.83	1.56	1.92	2.31
Fine-tuned Pei	2.6	2.08	1.29	1.54	4.26
Fine-tuned Llama-3.2-1B-Instruct	1.85	2.14	2.05	2.06	1.79
Prompted GPT	1.07	1.42	1.90	1.16	0.75
Fine-tuned GPT	1.04	1.24	1.36	0.99	1.16
Fine-tuned GPT (Rounded)	0.99	1.05	1.42	0.98	0.94