notesum.ai

Published at November 7

cs.CR

cs.AI

Released Date: November 7, 2024

Authors: Dylan Manuel¹, Nafis Tanveer Islam¹, Joseph Khoury², Ana Nunez¹, Elias Bou-Harb², Peyman Najafirad¹

Aff.: ¹Secure AI an Autonomy Laboratory, University of Texas at San Antonio; ²Louisiana State University

Model	Training	Acc	Pre.	Rec.	F1	Acc.V	Acc.B
CodeLLaMa	-	0.56	0.6	0.78	0.68	0.78	0.23
CodeLLaMa	DBVul	0.85	0.89	0.86	0.87	0.86	0.84
CodeGen2	-	0.59	0.65	0.83	0.73	0.83	0.13
CodeGen2	DBVul	0.91	0.93	0.94	0.94	0.94	0.86
Mistral	-	0.48	0.71	0.42	0.53	0.42	0.61
Mistral	DBVul	0.89	0.95	0.88	0.91	0.88	0.9
StarCoder	-	0.59	0.6	0.97	0.74	0.97	0.01
StarCoder	DBVul	0.89	0.91	0.93	0.92	0.93	0.80
LLaMa 3	-	0.57	0.7	0.68	0.69	0.68	0.34
LLaMa 3	DBVul	0.91	0.94	0.93	0.93	0.93	0.87