notesum.ai
Published at November 21RAG-Thief: Scalable Extraction of Private Data from Retrieval-Augmented Generation Applications with Agent-based Attacks
cs.CR
Released Date: November 21, 2024
Authors: Changyue Jiang1, Xudong Pan2, Geng Hong2, Chenfu Bao3, Min Yang2
Aff.: 1Fudan University, China; Shanghai Innovation Institute, China; 2Fudan University, China; 3Baidu Inc., China

| Datasets | Model | RAG-Thief | PIDE [18] | ||
|---|---|---|---|---|---|
| Untargeted Attack | Targeted Attack | Untargeted Attack | Targeted Attack | ||
| HealthCareMagic | ChatGPT-4 | 51% | 54% | 19% | 23% |
| Qwen2-72B-Instruct | 54% | 57% | 17% | 19% | |
| GLM-4-Plus | 51% | 55% | 17% | 21% | |
| Enron Email | ChatGPT-4 | 58% | 60% | 16% | 16% |
| Qwen2-72B-Instruct | 52% | 58% | 18% | 17% | |
| GLM-4-Plus | 53% | 56% | 17% | 17% | |
| Harry Potter | ChatGPT-4 | 69% | 77% | 9% | 35% |
| Qwen2-72B-Instruct | 73% | 79% | 9% | 30% | |
| GLM-4-Plus | 70% | 75% | 8% | 32% | |