notesum.ai

Published at December 10

cs.CL

Released Date: December 10, 2024

Authors: Zijian Chen¹, John-Michael Gamble¹, Micaela Jantzi², John P. Hirdes², Jimmy Lin¹

Aff.: ¹University of Waterloo; ²University of Waterloo, InterRAI Canada

Correct Level	Health Canada			RABBITS			Ontario Health
Correct Level	Fine-tuned 8B^*	Llama 3.1 70B	GPT-4o	Fine-tuned 8B^*	Llama 3.1 70B	GPT-4o	Fine-tuned 8B^*	Llama 3.1 70B
$\geq 5$	60.5%	60.3%	78.4%	26.4%	19.8%	39.4%	53.1%	49.4%
$\geq 4$	67.7%	64.7%	79.1%	32.9%	32.1%	47.8%	68.3%	67.5%
$\geq 3$	78.3%	74.6%	84.3%	43.5%	43.5%	55.4%	85.2%	83.2%
$\geq 2$	84.7%	80.7%	87.3%	46.7%	52.7%	64.4%	88.3%	88.0%
$\geq 1$	90.3%	87.1%	90.3%	62.8%	71.2%	81.8%	91.2%	89.8%