notesum.ai

Published at November 14

A Benchmark for Long-Form Medical Question Answering

cs.CL

cs.AI

Released Date: November 14, 2024

Authors: Pedram Hosseini¹, Jessica M. Sin², Bing Ren², Bryceton G. Thomas², Elnaz Nouri¹, Ali Farahanchi¹, Saeed Hassanpour³

Aff.: ¹Lavita AI; ²Dartmouth Hitchcock Medical Center; ³Dartmouth College

Arxiv: http://arxiv.org/abs/2411.09834v1

Level	Description
Basic	Medical questions in this category are basic and straightforward. The answers become apparent immediately upon reading the question or can be easily located through a simple Google or Internet search. Some questions may require a minor application of real-world knowledge, rephrasing, or expanding on the information to find the answer.
Intermediate	This category includes medical questions that are somewhat complicated, requiring a greater application of real-world knowledge. These questions tend to be detailed and may necessitate complex paraphrasing or simplification for clearer understanding. They can involve practical situations that require emotional support, psychological evaluations, and ethical considerations. Typical questions in this category might be similar to those found in USMLE exams. Furthermore, questions in this category might be based on vague symptom descriptions, making the diagnosis challenging, though they do not yet involve the most intricate scenarios of medical practice.
Advanced	This category involves complex medical questions that require extensive and detailed medical knowledge. Questions at this level are often lengthy and intricate, relating to real-world scenarios that include actual medical cases with challenging diagnoses and treatments. The symptom descriptions can be highly vague, adding to the diagnostic challenge. These questions necessitate advanced multi-step thinking and decision-making, often involving new technologies, recent medical publications, or current global health issues like pandemics. This level demands a high level of decision-making skills, the ability to choose the best available option, and the demonstration of humane care, pushing the boundaries of medical expertise and ethical considerations.