notesum.ai

Published at November 21

Next-Generation Phishing: How LLM Agents Empower Cyber Attackers

cs.CR

cs.AI

Released Date: November 21, 2024

Authors: Khalifa Afane¹, Wenqi Wei¹, Ying Mao¹, Junaid Farooq², Juntao Chen¹

Aff.: ¹Department of Computer and Information Sciences, Fordham University, New York, NY, USA; ²Department of Electrical and Computer Engineering, University of Michigan-Dearborn, Dearborn, MI, USA

Arxiv: http://arxiv.org/abs/2411.13874v1

Original Emails
Detector	TP	TN	FP	FN	Accuracy	Precision	Recall	F1 Score
Gmail Spam Filter	573	589	27	11	96.83%	95.50%	98.12%	96.79%
Proofpoint	558	592	41	8	96.16%	93.00%	98.59%	95.71%
SpamAssassin	574	576	26	24	95.83%	95.67%	95.99%	95.83%
Naive Bayes	559	567	41	33	93.80%	93.17%	94.43%	93.79%
SVM	542	555	58	45	91.42%	90.33%	92.33%	91.32%
Logistic Regression	554	569	46	31	93.58%	92.33%	94.70%	93.50%
Zero Shot Rephrased Emails
Gmail Spam Filter	573	559	27	41	94.33%	95.50%	93.32%	94.40%
Proofpoint	558	554	42	46	92.67%	93.00%	92.38%	92.69%
SpamAssassin	574	545	26	55	93.25%	95.67%	91.26%	93.41%
Naive Bayes	559	518	41	82	89.75%	93.17%	87.21%	90.09%
SVM	542	533	58	67	89.58%	90.33%	89.00%	89.66%
Logistic Regression	554	515	46	85	89.08%	92.33%	86.70%	89.43%
Few Shot Rephrased Emails
Gmail Spam Filter	573	483	27	117	88.00%	95.50%	83.04%	88.84%
Proofpoint	558	505	42	95	88.58%	93.00%	85.45%	89.07%
SpamAssassin	574	465	26	135	86.50%	95.67%	80.96%	87.70%
Naive Bayes	559	418	41	182	81.42%	93.17%	75.44%	83.37%
SVM	542	421	58	179	80.25%	90.33%	75.17%	82.06%
Logistic Regression	554	460	46	140	84.50%	92.33%	79.83%	85.63%