notesum.ai

Published at December 5

On the Lack of Robustness of Binary Function Similarity Systems

cs.CR

cs.LG

Released Date: December 5, 2024

Authors: Gianluca Capozzi¹, Tong Tang², Jie Wan², Ziqi Yang², Daniele Cono D'Elia¹, Giuseppe Antonio Di Luna¹, Lorenzo Cavallaro³, Leonardo Querzoni¹

Aff.: ¹Sapienza University of Rome; ²Zhejiang University; ³University College London

Arxiv: http://arxiv.org/pdf/2412.04163v1

		Models
		Gemini		GMN		ZEEK		BinFinder		SAFE		jTrans		Trex		PalmTree		AVG
	$\|P\|$	128	512	128	512	128	512	128	512	128	512	128	512	128	512	128	512	128	512
	wASR	48.25	25.10	64.68	40.15	26.07	9.34	88.95	71.37	64.80	40.02	45.25	23.35	54.85	30.48	63.60	30.24	57.06	33.76
	INIT	17.20	5.70	15.80	3.90	15.0	3.47	12.12	3.11	13.90	4.0	23.90	4.80	12.10	3.30	14.97	4.11	15.62	4.05
	ASR	71.90	45.10	78.60	59.80	43.16	19.49	94.39	80.96	80.30	59.10	80.20	49.10	68.80	44.60	78.04	47.19	74.42	50.67
	M-Instrs	173.62	170.18	214.53	210.35	94.86	101.21	155.21	154.78	123.74	128.05	77.29	84.46	112.70	112.76	193.41	182.05	143.17	142.98
@1	M-Nodes	13.83	15.23	18.87	18.35	11.18	10.80	33.28	33.0	15.72	15.72	11.95	12.66	11.44	11.36	18.28	18.62	16.82	16.97
	INIT	9.40	2.30	8.10	1.90	6.53	0.71	7.82	1.80	8.60	1.40	8.30	0.80	8.20	1.60	7.98	1.40	8.12	1.49
	ASR	57.0	29.20	71.10	47.30	31.94	11.33	91.68	75.55	71.40	46.30	58.90	28.0	60.40	34.40	70.06	34.97	64.06	38.38
	M-Instrs	173.68	173.84	213.81	214.23	96.81	108.93	155.94	156.73	126.32	128.05	84.68	94.18	112.83	114.63	192.08	173.26	144.52	145.67
@2	M-Nodes	14.68	16.26	18.63	18.68	10.35	11.50	33.40	33.46	15.89	15.72	12.65	13.61	11.30	11.13	18.09	18.08	16.87	17.30
	INIT	2.70	0.50	3.30	0.40	1.84	0	4.91	0.80	3.10	0.3	1.30	0.1	4.10	0.50	2.89	0.10	3.02	0.34
	ASR	38.0	15.70	60.40	32.30	18.57	4.29	87.37	67.33	60.10	32.30	27.30	10.40	49.50	24.10	57.58	22.14	49.85	26.07
	M-Instrs	177.87	181.08	214.63	224.91	93.72	108.83	157.19	159.0	129.75	131.05	96.62	105.67	115.36	114.20	189.60	175.48	146.84	150.01
@3	M-Nodes	15.76	17.45	18.92	19.17	10.26	11.14	33.67	33.88	15.84	15.88	13.86	15.56	11.34	11.20	18.15	19.18	17.23	17.93
	INIT	0.90	0.20	1.60	0	0.31	0	3.21	0.40	1.40	0	0.30	0.10	2.60	0.30	1.20	0	1.44	0.13
	ASR	26.10	10.40	48.60	21.20	10.61	2.24	82.36	61.62	47.40	22.40	14.60	5.9	40.70	18.80	48.70	16.63	39.88	19.90
	M-Instrs	190.36	192.19	219.14	239.85	104.95	124.5	159.24	160.23	136.29	140.71	109.49	118.15	116.21	117.37	192.21	176.20	153.49	158.65
@4	M-Nodes	16.57	18.25	19.48	19.49	10.99	13.45	34.19	34.12	16.05	16.40	14.71	16.24	11.42	11.53	18.37	19.39	17.72	18.60