notesum.ai

Published at October 20

cs.RO

cs.AI

cs.CL

cs.CV

cs.LG

Released Date: October 20, 2024

Authors: Daniel Ayzenshteyn¹, Roy Weiss¹, Yisroel Mirsky¹

Aff.: ¹Ben-Gurion University of the Negev, Israel

Model	Open Source?	Context Window	MMLU (5-shot)
GPT-4o [21]	✗	128K	88.7%
Claude Sonnet 3.5 [3]	✗	200K	90.4%
Gemini Pro 1.5 [17]	✗	2M	85.9%
LLaMA 3.1 70B Instruct ²²2Quantized [8]	✓	128K	81.8%