notesum.ai

Published at November 21

GMAI-VL & GMAI-VL-5.5M: A Large Vision-Language Model and A Comprehensive Multimodal Dataset Towards General Medical AI

cs.CV

Released Date: November 21, 2024

Authors: Tianbin Li¹, Yanzhou Su¹, Wei Li², Bin Fu², Zhe Chen³, Ziyan Huang², Guoan Wang⁴, Chenglong Ma⁵, Ying Chen⁶, Ming Hu⁷, Yanjun Li⁴, Pengcheng Chen⁸, Xiaowei Hu¹, Zhongying Deng⁹, Yuanfeng Ji¹⁰, Jin Ye⁷, Yu Qiao, Junjun He¹

Aff.: ¹Shanghai AI Laboratory; ²Shanghai AI Laboratory, Shenzhen Institute of Advanced Technology (SIAT), Chinese Academy of Sciences; ³Shanghai AI Laboratory, Nanjing University; ⁴Shanghai AI Laboratory, East China Normal University; ⁵Shanghai AI Laboratory, Fudan University; ⁶Shanghai AI Laboratory, Xiamen University; ⁷Shanghai AI Laboratory, Monash University; ⁸Shanghai AI Laboratory, University of Washington; ⁹Shanghai AI Laboratory, University of Cambridge; ¹⁰Stanford University

Arxiv: http://arxiv.org/abs/2411.14522v1

Model	VQA-RAD	SLAKE	PMC-VQA	Avg.
Med-Flamingo [54]	45.4	43.5	23.3	37.4
RadFM [70]	50.6	34.6	25.9	37.0
LLAVA-Med-7B [39]	51.4	48.6	24.7	41.6
Qwen-VL-Chat [6]	47.0	56.0	36.6	46.5
Yi-VL-34B [77]	53.0	58.9	39.5	50.5
LLAVA-v1.6-7B [46]	52.6	57.9	35.5	48.7
LLAVA-v1.6-13B [46]	55.8	58.9	36.6	50.8
LLAVA-v1.6-34B [46]	58.6	67.3	44.4	56.8
HuatuoGPT-Vision-7B [14]	63.8	74.5	52.7	63.7
GMAI-VL(w/o our data)	62.3	66.3	39.0	55.9
GMAI-VL(ours)	66.3	72.9	54.3	64.5