notesum.ai

Published at November 1

InterTrans: Leveraging Transitive Intermediate Translations to Enhance LLM-based Code Translation

cs.SE

cs.AI

Released Date: November 1, 2024

Authors: Marcos Macedo¹, Yuan Tian¹, Pengyu Nie², Filipe R. Cogo³, Bram Adams¹

Aff.: ¹Queen's University, Kingston, ON, Canada; ²University of Waterloo, Waterloo, ON, Canada; ³Huawei Canada, Kingston, ON, Canada

Arxiv: http://arxiv.org/abs/2411.01063v1

Refer to caption

Dataset

Source language

Total samples

CA@K (percentage)

Code Llama

Magicoder

StarCoder2

Direct

(CA@1)

Direct

(CA@10)

Inter

Trans

Abs.

diff.

Rel.

diff.

Direct

(CA@1)

Direct

(CA@10)

Inter

Trans

Abs.

diff.

Rel.

diff.

Direct

(CA@1)

Direct

(CA@10)

Inter

Trans

Abs.

diff.

Rel.

diff.

CodeNet

C++

175

32.0

42.9

61.1

18.3

42.7

50.3

50.9

88.0

37.1

73.0

29.1

40.0

81.7

41.7

104.3

Go

175

30.3

34.3

61.1

26.9

78.3

50.9

53.1

85.7

32.6

61.3

45.7

50.3

85.1

34.9

69.3

Java

175

25.7

38.9

55.4

16.6

42.6

45.1

45.7

85.1

39.4

86.2

36.6

41.1

85.7

44.6

108.3

JavaScript

175

22.3

33.7

64.6

30.9

91.5

50.9

50.9

87.4

36.6

71.9

24.0

25.7

82.9

57.1

222.2

Python

175

14.3

19.4

57.1

37.7

194.1

41.1

42.3

91.4

49.1

116.2

38.3

44.0

87.4

43.4

98.7

Rust

175

29.7

38.3

65.1

26.9

70.1

50.9

51.4

86.3

34.9

67.8

36.0

45.1

83.4

38.3

84.8

Total/Average

1,050

25.7

34.6

60.8

26.2

75.8

48.2

49.0

87.3

38.3

78.1

35.0

41.0

84.4

43.3

105.6

HumanEval-X

C++

175

70.3

78.9

91.4

12.6

15.9

73.1

74.3

97.7

23.4

31.5

61.1

66.9

86.3

19.4

29.1

Go

175

64.0

71.4

90.3

18.9

26.4

62.9

64.0

98.3

34.3

53.6

52.0

55.4

83.4

28.0

50.5

Java

175

58.3

68.0

87.4

19.4

28.6

65.7

67.4

93.1

25.7

38.1

46.9

48.6

86.3

37.7

77.6

JavaScript

175

57.1

73.1

93.1

20.0

27.3

60.6

60.6

96.0

35.4

58.5

44.0

44.0

80.6

36.6

83.1

Python

175

53.7

64.6

82.3

17.7

27.4

61.7

62.9

89.7

26.9

42.7

36.6

36.6

77.1

40.6

110.9

Rust

175

59.4

72.0

93.7

21.7

30.2

71.4

72.0

97.7

25.7

35.7

52.6

54.3

81.1

26.9

49.5

Total/Average

1,050

60.5

71.3

89.7

18.4

25.8

65.9

66.9

95.4

28.6

42.7

48.9

51.0

82.5

31.5

61.9

TransCoder

C++

946

73.9

75.9

93.2

17.3

22.8

67.9

67.9

92.7

24.8

36.6

63.5

65.2

93.8

28.5

43.8

Java

931

77.7

79.5

94.8

15.4

19.3

77.4

77.4

91.9

14.5

18.7

79.3

79.9

95.1

15.1

19.0

Python

949

67.3

69.3

91.6

22.2

32.1

33.5

33.5

87.8

54.3

161.9

73.9

74.6

92.7

18.1

24.3

Total/Average

2,826

72.9

74.9

93.2

18.3

24.5

59.5

59.5

90.8

31.3

52.6

72.2

73.2

93.8

20.6

28.2