notesum.ai

Published at October 30

SoftCTRL: Soft conservative KL-control of Transformer Reinforcement Learning for Autonomous Driving

cs.RO

cs.AI

Released Date: October 30, 2024

Authors: Minh Tri Huynh¹, Duc Dung Nguyen¹

Aff.: ¹AITech Lab., Ho Chi Minh City University of Technology (HCMUT), Vietnam

Arxiv: http://arxiv.org/abs/2410.22752v1

Refer to caption

Method	Imitation			Collisions		Discomfort		Failure
Method	$\mu_{\text{ADE}}$	$\mu_{\text{D2R}}$	D2R ( $\geq 4$ m)	$\mu_{\text{CL}}$	CL ( $\geq 1$ )	$\mu_{\text{ACC}}$	ACC ( $\geq 2$ m/s²)	D2R+CL
BC-perturb	$\mathbf{10.63\pm\scriptstyle 0.06}$	$\mathbf{1\pm\scriptstyle 0}$	$8.67\pm\scriptstyle 0.58$	$8\pm\scriptstyle 0.1$	$25.33\pm\scriptstyle 1.53$	$0.05\pm\scriptstyle 0.00$	$3.86\pm\scriptstyle 0.26$	$34\pm\scriptstyle 2$
BPTT	$19.9\pm\scriptstyle 1.2$	$1.4\pm\scriptstyle 0.3$	$\mathbf{6\pm\scriptstyle 0.5}$	$10\pm\scriptstyle 0.82$	$49\pm\scriptstyle 3.43$	$0.04\pm\scriptstyle 0.01$	$3.67\pm\scriptstyle 0.12$	$55\pm\scriptstyle 4$
PPO	$45.5\pm\scriptstyle 12.91$	$1.63\pm\scriptstyle 0.4$	$15.33\pm\scriptstyle 5.13$	$20.87\pm\scriptstyle 1.14$	$139\pm\scriptstyle 27.5$	$0.03\pm\scriptstyle 0.01$	$1.06\pm\scriptstyle 0.43$	$154.33\pm\scriptstyle 22.9$
SAC	$24.3\pm\scriptstyle 2.05$	$2.43\pm\scriptstyle 0.15$	$21.33\pm\scriptstyle 2.08$	$1.87\pm\scriptstyle 0.25$	$27\pm\scriptstyle 5.29$	$0.09\pm\scriptstyle 0.04$	$13.61\pm\scriptstyle 10.64$	$48.33\pm\scriptstyle 6.66$
SAC-ExKL	$21.03\pm\scriptstyle 1.3$	$2.43\pm\scriptstyle 0.15$	$15.33\pm\scriptstyle 2.31$	$1.03\pm\scriptstyle 0.61$	$19.33\pm\scriptstyle 7.57$	$\mathbf{0.02\pm\scriptstyle 0.00}$	$\mathbf{0.31\pm\scriptstyle 0.06}$	$34.67\pm\scriptstyle 6.43$
SAC-ImKL	$22.57\pm\scriptstyle 0.71$	$2.1\pm\scriptstyle 0.1$	$13.33\pm\scriptstyle 2.52$	$\mathbf{0.83\pm\scriptstyle 0.06}$	$\mathbf{15\pm\scriptstyle 2.65}$	$0.05\pm\scriptstyle 0.01$	$3.83\pm\scriptstyle 1.89$	$\mathbf{28.33\pm\scriptstyle 0.58}$