notesum.ai

Published at November 13

The Limited Impact of Medical Adaptation of Large Language and Vision-Language Models

cs.CL

cs.AI

cs.LG

Released Date: November 13, 2024

Authors: Daniel P. Jeong¹, Pranav Mani², Saurabh Garg³, Zachary C. Lipton¹, Michael Oberst⁴

Aff.: ¹Machine Learning Department, Carnegie Mellon University; ²Abridge AI; ³Unknown Affiliation; ⁴Department of Computer Science, Johns Hopkins University

Arxiv: http://arxiv.org/abs/2411.08870v1

Model Class	General Domain	Medical Domain	Medical Adaptation Corpora
LLM	Llama-3-70B-Instruct (Meta, 2024)	Med42-v2-70B (Christophe et al., 2024b)	Medical QA Datasets (e.g., MedQA, MedMCQA)
			Medical Instruction 120k (Altaf, 2023)
			OpenGPT (OpenChat) (Wang et al., 2024)
			StackExchange (Lambert et al., 2023)
			Medical Flashcards (Han et al., 2023)
	Llama-3-70B-Instruct (Meta, 2024)	OpenBioLLM-70B (Pal and Sankarasubbu, 2024)	Undisclosed
	Llama-2-70B (Touvron et al., 2023b)	MediTron-70B (Chen et al., 2023)	Clinical Practice Guidelines (e.g., CDC, WHO)
	Llama-2-70B (Touvron et al., 2023b)	MediTron-70B (Chen et al., 2023)	PubMed Articles (S2ORC; Lo et al., 2020)
	Llama-2-70B (Touvron et al., 2023b)	Clinical-Camel-70B (Toma et al., 2023)	ShareGPT
			20k PubMed Articles Published Before 2021
			Random 4k Subset of MedQA (Jin et al., 2020)
	Llama-2-70B (Touvron et al., 2023b)	Med42-v1-70B (Christophe et al., 2024a)	Medical QA Datasets (e.g., MedQA, MedMCQA)
			OpenGPT (OpenChat) (Wang et al., 2024)
			StackExchange (Lambert et al., 2023)
			Medical Flashcards (Han et al., 2023)
			CORD-19 (Wang et al., 2020)
	Llama-3-8B-Instruct (Meta, 2024)	Med42-v2-8B (Christophe et al., 2024b)	Medical QA Datasets (e.g., MedQA, MedMCQA)
			Medical Instruction 120k (Altaf, 2023)
			OpenGPT (OpenChat) (Wang et al., 2024)
			StackExchange (Lambert et al., 2023)
			Medical Flashcards (Han et al., 2023)
	Llama-3-8B (Meta, 2024)	OpenBioLLM-8B (Pal and Sankarasubbu, 2024)	Undisclosed
	Llama-2-7B (Touvron et al., 2023b)	MediTron-7B (Chen et al., 2023)	Clinical Practice Guidelines (e.g., CDC, WHO)
	Llama-2-7B (Touvron et al., 2023b)	MediTron-7B (Chen et al., 2023)	PubMed Articles (S2ORC; Lo et al., 2020)
	Mistral-7B-Instruct-v0.1 (Jiang et al., 2023)	BioMistral-7B (Labrak et al., 2024)	PubMed Articles (PMC Open Access Subset)
	Llama-2-7B-Chat (Touvron et al., 2023b)	BioMedGPT-LM-7B (Luo et al., 2023)	PubMed Articles (S2ORC; Lo et al., 2020)
VLM	LLaVA-v0-7B (Liu et al., 2023)	LLaVA-Med-7B (Li et al., 2023a)	PubMed Articles (PMC-15M; Zhang et al., 2023)
	Open-Flamingo-9B (Awadalla et al., 2023)	Med-Flamingo-9B (Moor et al., 2023b)	Medical Textbooks (MTB; Moor et al., 2023b)
	Open-Flamingo-9B (Awadalla et al., 2023)	Med-Flamingo-9B (Moor et al., 2023b)	PubMed Articles (PMC-OA; Lin et al., 2023)