notesum.ai

Published at October 22

cs.SD

cs.AI

cs.CL

eess.AS

Released Date: October 22, 2024

Authors: Itay Nakash¹, George Kour¹, Guy Uziel¹, Ateret Anaby-Tavor¹

Aff.: ¹IBM Research AI

Method	Mixtral	Llama-3.1	Llama-3	GPT-4o-mini	Mean
IPI	30.3	70.5	57.5	9.3	41.9
IPI+Unfamiliar FITD (Mean)	37.0 (9.0)	67.7 (7.8)	54.2 (2.7)	29.0 (6.2)	47.0
IPI+Unfamiliar FITD (Calculator)	49.3	78.9	59.0	40.5	56.9
IPI+Familiar FITD (Mean)	66.2 (7.5)	91.6 (4.1)	93.8 (2.1)	54.1 (5.4)	76.4
IPI+Familiar FITD (Calculator)	71.8	96.0	93.9	65.5	81.8
IPI+TI	98.3	77.6	79.7	95.4	87.8
IPI+FITD+TI	99.2	97.0	96.7	95.4	97.1
IPI+FITD+HTI	96.4	96.5	95.9	83.3	93.0