notesum.ai
Published at November 11More Expressive Attention with Negative Weights
cs.CL
cs.AI
cs.LG
Released Date: November 11, 2024
Authors: Ang Lv, Ruobing Xie, Shuaipeng Li, Jiayi Liao, Xingwu Sun, Zhanhui Kang, Rui Yan

| Model | ARC-E | ARC-C | PIQA | SST-2 | MNLI | MRPC | QQP | RTE | Avg. |
|---|---|---|---|---|---|---|---|---|---|
| Transformer | 42.34 | 19.54 | 57.73 | 51.72 | 33.21 | 68.63 | 36.82 | 51.99 | 45.24 |
| Cogformer | 43.90 | 19.54 | 59.09 | 54.59 | 34.12 | 68.38 | 37.00 | 52.71 | 46.16 |