notesum.ai
Published at October 20A Survey of Hallucination in Large Visual Language Models
cs.CV
cs.AI
cs.GR
Released Date: October 20, 2024
Authors: Wei Lan, Wenyi Chen, Qingfeng Chen, Shirui Pan, Huiyu Zhou, Yi Pan
| Correction Method | Goal Scene | Train | Address |
|---|---|---|---|
| Text Shearing | Noise data; Mismatched data; long-tail phenomenon | Free | https://github.com/lyq312318224/MLLMs-Augmented |
| CIT | Hallucinations of Object; Over-confidence | Free | – |
| LRV-Instruction | Hallucinations of Object; Over-confidence | Free | https://fuxiaoliu.github.io/LRV/ |
| HalluciDoctor | Hallucinations of Object | Free | https://github.com/Yuqifan1117/HalluciDoctor/ |
| COMM | Visual details | Need | – |
| MOF | Visual details | Need | – |
| DualFocus | Visual details | Need | https://github.com/InternLM/InternLM-XComposer/blob/main/projects/DualFocus |
| VTPrompt | Visual Prompt; Textual Prompt | Free | https://github.com/jiangsongtao/VTprompt |
| HACL | Hallucinations of Object | Need | – |
| Woodpecker | Hallucinations of Object | Free | https://github.com/BradyFU/Woodpecker |
| LURE | Co-occurrence phenomenon; long-tail phenomenon | Need | https://github.com/YiyangZhou/LURE |
| Volcano | Iterative self-revision | Need | https://github.com/kaistAI/Volcano |
| Factually Augmented RLHF | Human preferences | Need | https://llava-rlhf.github.io/ |
| RLHF-V | Human preferences | Need | https://rlhf-v.github.io/ |
| HA-DPO | Human preferences | Need | – |
| Fact | CoT | Need | – |
| Cantor | CoT | Free | https://ggg0919.github.io/cantor/ |
| OPERA | Knowledge aggregation pattern | Free | https://github.com/shikiw/OPERA |
| VIGC | long-tail phenomenon | Need | https://opendatalab.github.io/VIGC/ |
| Halle-Switch | Parametric knowledge control | Need | https://github.com/bronyayang/HallE_Switch |
| Pensieve | Perception module error bets | Free | https://github.com/DingchenYang99/Pensieve |
| EFUF | Text-image similarity | Need | – |