notesum.ai
Published at November 19Guide-to-Explain for Controllable Summarization
cs.CL
cs.AI
Released Date: November 19, 2024
Authors: Sangwon Ryu1, Heejin Do1, Daehee Kim1, Yunsu Kim2, Gary Geunbae Lee3, Jungseul Ok3
Aff.: 1Graduate School of Artificial Intelligence, POSTECH, South Korea; 2aiXplain Inc., Los Gatos, CA, USA; 3Department of Computer Science and Engineering, POSTECH, South Korea

| Model | Extractiveness ( / ) | Length ( / ) | Topic () | Speaker () | ||||||
| normal | high | fully | avg | short | normal | long | avg | |||
| Llama3-70B-Iter | 31.78% / 8.13 | 43.59% / 8.40 | 8.16% / 5.39 | 29.63% / 7.59 | 12.00% / | 93.75% / 6.00 | 98.00% / | 81.79% / 6.00 | 0.00% / 0.01 | 0.00% / 0.00 |
| Llama3-70B-AIS | 14.41% / 5.91 | 23.08% / 5.31 | 0.00% / 3.72 | 13.27% / 5.50 | 0.00% / 1.25 | 62.05% / 5.70 | 92.00% / 9.33 | 57.10% / 5.62 | 0.00% / 0.02 | 0.00% / 0.00 |
| Llama3-70B-GTE | 0.00% / 2.31 | 0.00% / 2.56 | 4.08% / 3.64 | 0.61% / 2.49 | 0.00% / 1.00 | 36.61% / 4.73 | 80.00% / 5.70 | 37.65% / 4.53 | 0.00% / 0.01 | 0.00% / 0.00 |
| GPT-4o-Iter | 79.24% / 4.36 | 82.05% 3.67 | 59.18% / 1.00 | 76.54% / 4.00 | 6.00% / | 98.21% / | 100.00% / | 84.26% / | 0.31% / 0.01 | 0.00% / 0.00 |
| GPT-4o-AIS | 84.75% / 4.00 | 87.18% 1.50 | 53.06% 5.10 | 80.25% / 4.32 | 2.00% / 4.50 | 96.43% / | 100.00% / | 82.41% / 4.50 | 0.00% / 0.01 | 0.00% / 0.00 |
| GPT-4o-GTE | 17.80% / 7.94 | 25.64% / 7.92 | 8.16% / 4.58 | 17.28% / 7.53 | 0.00% / 1.40 | 9.82% / 2.75 | 44.00% / 4.21 | 13.58% / 2.90 | 0.00% / 0.02 | 0.00% / 0.00 |