notesum.ai

Published at November 17

On-Board Vision-Language Models for Personalized Autonomous Vehicle Motion Control: System Design and Real-World Validation

cs.AI

cs.RO

Released Date: November 17, 2024

Authors: Can Cui¹, Zichong Yang¹, Yupeng Zhou¹, Juntong Peng¹, Sung-Yeon Park¹, Cong Zhang¹, Yunsheng Ma¹, Xu Cao², Wenqian Ye³, Yiheng Feng¹, Jitesh Panchal¹, Lingxi Li¹, Yaobin Chen¹, Ziran Wang¹

Aff.: ¹Purdue University; ²University of Illinois Urbana-Champaign; ³University of Virginia

Arxiv: http://arxiv.org/abs/2411.11913v1

Driving Scenario

Model

Safety Metrics

Comfort Metrics

Time Efficiency

Alignment

Driving Score

\uparrow

Time to

Collision(

s

)

\uparrow

SV_{x}

(

m^{2}/s^{2}

)

\downarrow

SV_{y}

(

10^{-2}m^{2}/s^{2}

)

\downarrow

|\bar{\alpha_{x}}|

(

m/s^{2}

)

\downarrow

|\bar{J_{x}}|

(

m/s^{3}

)

\downarrow

|\bar{\alpha_{y}}|

(

10^{-1}m/s^{2}

)

\downarrow

|\bar{J_{y}}|

(

m/s^{3}

)

\downarrow

LLM

Latency(

s

)

\downarrow

Command

Alignment

\uparrow

Scenario

Alignment

\uparrow

Acceleration

Baseline

2.44

28.8

0.36

0.78

3.27

0.46

0.44

92.0

60.0

75.6

GPT-4o

2.52

30.0

0.39

0.83

3.40

0.52

5.82

92.9

71.3

76.4

Ours

2.46

30.8

0.39

0.81

3.07

0.53

0.81

1.98

96.3

60.9

76.5

Lane Change

Baseline

2.44

3.91

1.65

0.37

3.14

0.88

1.01

88.5

60.0

74.5

GPT-4o

2.71

3.88

2.23

0.53

4.38

1.13

1.39

4.84

90.4

88.6

78.4

Ours

2.15

4.07

2.15

0.41

3.35

0.98

1.02

1.83

92.2

71.9

77.5

Left Turn

Baseline

1.12

7.52

0.22

1.81

1.29

1.36

88.0

60.0

70.4

GPT-4o

0.93

11.5

0.29

2.75

2.11

2.40

5.23

91.3

85.0

71.4

Ours

0.94

6.74

0.19

1.67

1.33

1.32

1.64

90.2

67.8

74.4