notesum.ai

Published at November 12

cs.CL

cs.AI

I.2.7

Released Date: November 12, 2024

Authors: Shuai Niu¹, Jing Ma¹, Liang Bai², Zhihua Wang³, Yida Xu¹, Yunya Song¹, Xian Yang⁴

Aff.: ¹Hong Kong Baptist University; ²Shanxi University; ³Shanghai Institute for Advanced Study of Zhejiang University; ⁴The University of Manchester

Models	Modality		Micro			Macro
Models	Lab	Note	Precision	Recall	F1	Precision	Recall	F1
MIMIC-III
Flan-T5-small 60M Chung et al. (2024)		✓	0.5845	0.6654	0.6223	0.5687	0.6272	0.5906
PROMPTEHR 75.2M Wang and Sun (2022)		✓	0.5906	0.6621	0.6243	0.5795	0.6217	0.5909
LLaMA 7B Touvron et al. (2023)		✓	0.6225	0.6485	0.6352	0.6178	0.6114	0.6080
FROZEN 265M Tsimpoukelli et al. (2021)	✓	✓	0.6054	0.6495	0.6267	0.5921	0.6110	0.5944
EHR-KnowGen 77M Niu et al. (2024a)	✓	✓	0.6003	0.6548	0.6264	0.5854	0.6152	0.5945
Clinical CoT 60M Kwon et al. (2024)		✓	0.6223	0.6421	0.6324	0.6057	0.6013	0.5968
ClinRaGen 87M	✓	✓	0.6107	0.6752	0.6413	0.5998	0.6328	0.6117
MIMIC-IV
Flan-T5-small 60M Chung et al. (2024)		✓	0.6658	0.6961	0.6806	0.6441	0.6632	0.6507
PROMPTEHR 75.2M Wang and Sun (2022)		✓	0.6592	0.7052	0.6814	0.6417	0.6686	0.6506
LLaMA 7B Touvron et al. (2023)		✓	0.6872	0.6987	0.6923	0.6786	0.6676	0.6649
FROZEN 265M Tsimpoukelli et al. (2021)	✓	✓	0.6640	0.7038	0.6833	0.6493	0.6655	0.6528
EHR-KnowGen 77M Niu et al. (2024a)	✓	✓	0.6639	0.7025	0.6827	0.6462	0.6641	0.6515
Clinical CoT 60M Kwon et al. (2024)		✓	0.6745	0.7092	0.6914	0.6627	0.6728	0.6638
ClinRaGen 87M	✓	✓	0.7011	0.6971	0.6991	0.6874	0.6607	0.6697