notesum.ai
Published at November 12Deceiving Question-Answering Models: A Hybrid Word-Level Adversarial Approach
cs.CL
cs.AI
cs.LG
Released Date: November 12, 2024
Authors: Jiyao Li1, Mingze Ni1, Yongshun Gong2, Wei Liu1
Aff.: 1School of Computer Science, University of Technology Sydney; 2School of Software, Shandong University
| Datasets | Methods | Noun (%) | Verb (%) | Adj. (%) | Adv. (%) | Others (%) |
|---|---|---|---|---|---|---|
| SQuAD 1.1 | TASA | 35 | 12 | 13 | 36 | 4 |
| TMYC | 47 | 21 | 11 | 5 | 17 | |
| RobustQA | 34 | 13 | 22 | 16 | 15 | |
| Textfooler | 44 | 13 | 23 | 8 | 12 | |
| T3 | 60 | 17 | 6 | 7 | 10 | |
| QA-Attack | 34 | 9 | 18 | 3 | 36 | |
| BoolQ | TASA | – | – | - | – | - |
| TMYC | 14 | 19 | 12 | 35 | 20 | |
| RobustQA | 19 | 14 | 27 | 23 | 17 | |
| Textfooler | 41 | 15 | 27 | 7 | 10 | |
| T3 | 42 | 13 | 20 | 16 | 9 | |
| QA-Attack | 10 | 19 | 25 | 18 | 28 |