notesum.ai

Published at November 28

cs.CV

cs.AI

cs.CL

cs.LG

I.2.10; J.3; I.5.4

Released Date: November 28, 2024

Authors: Xi Zhang¹, Zaiqiao Meng¹, Jake Lever¹, Edmond S. L. Ho¹

Aff.: ¹Information Retrieval Group, AI4BioMed Lab, School of Computing Science

Metric	LLaVA-Med ${}^{\text{\textsection}}$	CheXagent ${}^{\text{\textdaggerdbl}}$	GPT-4V ${}^{\text{\textdaggerdbl}}$	Med-PaLM	LLaVA-Rad	MAIRA-1	Libra(%)
Lexical:
ROUGE-L	27.6	21.5	13.2	27.5	30.6	28.9	36.7 (19.9%)
BLEU-1	35.4	16.9	16.4	32.3	38.1	39.2	51.3 (30.9%)
BLEU-4	14.9	4.7	17.8	11.5	15.4	14.2	24.5 (37.6%)
METEOR	35.3	–	–	–	–	33.3	48.9 (38.5%)
Clinical:
RadGraph-F1	19.1	–	–	26.7	–	24.3	32.9 (23.2%)
RG ${}_{\text{ER}}$	23.8	20.5	13.2	–	29.4	29.6	37.6 (27.0%)
RadCliQ ${}_{\text{0}}$ ( $\downarrow$ )	3.3	–	–	–	–	3.1	2.7 (12.9%)
CheXbert vector	36.9	–	–	–	–	44.0	46.9 (6.59%)
CheXpert-F1:
Micro-F1-14	42.7	39.3	35.5	53.6	57.3	55.7	55.9 (-2.4%)
Macro-F1-14	26.9	24.7	20.4	39.8	39.5	38.6	40.4 (1.5%)
Micro-F1-5	43.9	41.2	25.8	57.9	57.4	56.0	60.1 (3.8%)
Macro-F1-5	36.3	34.5	19.6	51.6	47.7	47.7	53.8 (4.3%)