notesum.ai
Published at October 22Breaking ReAct Agents: Foot-in-the-Door Attack Will Get You In
cs.SD
cs.AI
cs.CL
eess.AS
Released Date: October 22, 2024
Authors: Itay Nakash1, George Kour1, Guy Uziel1, Ateret Anaby-Tavor1
Aff.: 1IBM Research AI

| Method | Mixtral | Llama-3.1 | Llama-3 | GPT-4o-mini | Mean |
| IPI | 30.3 | 70.5 | 57.5 | 9.3 | 41.9 |
| IPI+Unfamiliar FITD (Mean) | 37.0 (9.0) | 67.7 (7.8) | 54.2 (2.7) | 29.0 (6.2) | 47.0 |
| IPI+Unfamiliar FITD (Calculator) | 49.3 | 78.9 | 59.0 | 40.5 | 56.9 |
| IPI+Familiar FITD (Mean) | 66.2 (7.5) | 91.6 (4.1) | 93.8 (2.1) | 54.1 (5.4) | 76.4 |
| IPI+Familiar FITD (Calculator) | 71.8 | 96.0 | 93.9 | 65.5 | 81.8 |
| IPI+TI | 98.3 | 77.6 | 79.7 | 95.4 | 87.8 |
| IPI+FITD+TI | 99.2 | 97.0 | 96.7 | 95.4 | 97.1 |
| IPI+FITD+HTI | 96.4 | 96.5 | 95.9 | 83.3 | 93.0 |