notesum.ai

Published at October 21

Reducing Hallucinations in Vision-Language Models via Latent Space Steering

cs.CL

cs.AI

cs.DB

cs.LG

Released Date: October 21, 2024

Authors: Sheng Liu¹, Haotian Ye¹, Lei Xing¹, James Zou¹

Aff.: ¹Stanford University

Arxiv: https://arxiv.org/abs/2410.15778v2

Refer to caption

Model	LLaVA-1.5		InstructBLIP		Qwen-VL
Method	Accuracy $\uparrow$	F1 Score $\uparrow$	Accuracy $\uparrow$	F1 Score $\uparrow$	Accuracy $\uparrow$	F1 Score $\uparrow$
Vanilla	79.8	79.4	76.3	78.0	83.5	81.2
VCD	82.3	83.4	80.1	81.0	84.5	83.3
OPERA	84.2	83.7	79.6	80.9	84.3	82.6
VTI	86.5	85.9	81.8	83.2	85.2	84.1