notesum.ai
Published at November 12Fair Summarization: Bridging Quality and Diversity in Extractive Summaries
cs.CL
cs.AI
Released Date: November 12, 2024
Authors: Sina Bagheri Nezhad1, Sayan Bandyapadhyay1, Ameeta Agrawal1
Aff.: 1Portland State University, Department of Computer Science, Portland, Oregon, USA
| Model | SUPERT | BLANC | SummaQA | BARTScore | UniEval | F |
|---|---|---|---|---|---|---|
| Naive | 0.525 | 0.135 | 0.063 | -1.788 | 0.391 | 0.732 |
| NaiveFair | 0.526 | 0.137 | 0.065 | -1.776 | 0.386 | 1.000 |
| TextRank Vanilla | 0.527 | 0.108 | 0.081 | -1.852 | 0.401 | 0.727 |
| TextRank Cluster-A | 0.530 | 0.107 | 0.075 | -1.827 | 0.383 | 0.693 |
| TextRank Cluster-H | 0.530 | 0.107 | 0.077 | -1.922 | 0.387 | 0.709 |
| BERT-EXT Vanilla | 0.544 | 0.137 | 0.070 | -1.427 | 0.396 | 0.680 |
| BERT-EXT Cluster-A | 0.553 | 0.138 | 0.071 | -1.535 | 0.399 | 0.728 |
| BERT-EXT Cluster-H | 0.554 | 0.133 | 0.070 | -1.486 | 0.365 | 0.689 |
| ChatGPT-EXT | 0.668 | 0.140 | 0.065 | -0.642 | 0.434 | 0.698 |
| FairExtract (Ours) | 0.530 | 0.140 | 0.066 | -1.801 | 0.411 | 1.000 |
| FairGPT (Ours) | 0.644 | 0.139 | 0.075 | -0.821 | 0.418 | 1.000 |