notesum.ai

Published at November 19

Guide-to-Explain for Controllable Summarization

cs.CL

cs.AI

Released Date: November 19, 2024

Authors: Sangwon Ryu¹, Heejin Do¹, Daehee Kim¹, Yunsu Kim², Gary Geunbae Lee³, Jungseul Ok³

Aff.: ¹Graduate School of Artificial Intelligence, POSTECH, South Korea; ²aiXplain Inc., Los Gatos, CA, USA; ³Department of Computer Science and Engineering, POSTECH, South Korea

Arxiv: http://arxiv.org/abs/2411.12460v1

Model	Extractiveness ( $\downarrow$ / $\downarrow$ )				Length ( $\downarrow$ / $\downarrow$ )				Topic ( $\downarrow$ )	Speaker ( $\downarrow$ )
Model	normal	high	fully	avg	short	normal	long	avg	Topic ( $\downarrow$ )	Speaker ( $\downarrow$ )
Llama3-70B-Iter	31.78% / 8.13	43.59% / 8.40	8.16% / 5.39	29.63% / 7.59	12.00% / $\circlearrowright$	93.75% / 6.00	98.00% / $\circlearrowright$	81.79% / 6.00	0.00% / 0.01	0.00% / 0.00
Llama3-70B-AIS	14.41% / 5.91	23.08% / 5.31	0.00% / 3.72	13.27% / 5.50	0.00% / 1.25	62.05% / 5.70	92.00% / 9.33	57.10% / 5.62	0.00% / 0.02	0.00% / 0.00
Llama3-70B-GTE	0.00% / 2.31	0.00% / 2.56	4.08% / 3.64	0.61% / 2.49	0.00% / 1.00	36.61% / 4.73	80.00% / 5.70	37.65% / 4.53	0.00% / 0.01	0.00% / 0.00
GPT-4o-Iter	79.24% / 4.36	82.05% 3.67	59.18% / 1.00	76.54% / 4.00	6.00% / $\circlearrowright$	98.21% / $\circlearrowright$	100.00% / $\circlearrowright$	84.26% / $\circlearrowright$	0.31% / 0.01	0.00% / 0.00
GPT-4o-AIS	84.75% / 4.00	87.18% 1.50	53.06% 5.10	80.25% / 4.32	2.00% / 4.50	96.43% / $\circlearrowright$	100.00% / $\circlearrowright$	82.41% / 4.50	0.00% / 0.01	0.00% / 0.00
GPT-4o-GTE	17.80% / 7.94	25.64% / 7.92	8.16% / 4.58	17.28% / 7.53	0.00% / 1.40	9.82% / 2.75	44.00% / 4.21	13.58% / 2.90	0.00% / 0.02	0.00% / 0.00