notesum.ai

Published at November 27

Critic-V: VLM Critics Help Catch VLM Errors in Multimodal Reasoning

cs.CV

cs.CL

Released Date: November 27, 2024

Authors: Di Zhang¹, Jingdi Lei², Junxian Li³, Xunzhi Wang⁴, Yujie Liu⁵, Zonglin Yang⁶, Jiatong Li⁷, Weida Wang⁸, Suorong Yang⁹, Jianbo Wu¹⁰, Peng Ye¹, Wanli Ouyang¹¹, Dongzhan Zhou¹¹

Aff.: ¹Fudan University; ²Beijing Institute of Technology; ³Shanghai Jiaotong University; ⁴Nankai University; ⁵Shanghai University; ⁶Nanyang Technological University; ⁷Hong Kong Polytechnic University; ⁸Tongji University; ⁹Nanjing University; ¹⁰University of California, Merced; ¹¹Shanghai Artificial Intelligence Laboratory

Arxiv: http://arxiv.org/abs/2411.18203v1

Model	Benchmarks
Model	RealWorldQA [53]	MMStar [6]	MMBench [30]	SEEDBench [23]	ScienceQA [32]	MMT-Bench [58]
LLaVA-V1.5-7B	50.7	32.2	68.4	65.6	60.8	36.0
+POVID [66]	51.8	33.6	71.6	65.4	65.0	33.4
+CSR [67]	51.8	32.4	70.6	65.4	66.0	33.2
+SIMA [50]	49.3	32.6	70.6	65.2	64.2	34.0
+SCL [14]	53.2	35.8	70.8	68.6	67.8	39.6
+Critic-V(Ours)	63.5	38.4	73.8	70.1	65.2	49.7