Research on Anthropomorphic Obstacle Avoidance Trajectory Planning for Adaptive Driving Scenarios Based on Inverse Reinforcement Learning Theory

Jian Wu; Yang Yan; Yulong Liu; Yahui Liu

doi:10.1016/j.eng.2023.07.018

PDF(3076 KB)

Engineering ›› 2024, Vol. 33 ›› Issue (2) : 133-145. DOI: 10.1016/j.eng.2023.07.018

Research

Article

Research on Anthropomorphic Obstacle Avoidance Trajectory Planning for Adaptive Driving Scenarios Based on Inverse Reinforcement Learning Theory

Jian Wu^a ,
Yang Yan^a ,
Yulong Liu^b ,
Yahui Liu^b^,^*

Author information +

History +

Abstract

The forward design of trajectory planning strategies requires preset trajectory optimization functions, resulting in poor adaptability of the strategy and an inability to accurately generate obstacle avoidance trajectories that conform to real driver behavior habits. In addition, owing to the strong time-varying dynamic characteristics of obstacle avoidance scenarios, it is necessary to design numerous trajectory optimization functions and adjust the corresponding parameters. Therefore, an anthropomorphic obstacle-avoidance trajectory planning strategy for adaptive driving scenarios is proposed. First, numerous expert-demonstrated trajectories are extracted from the HighD natural driving dataset. Subsequently, a trajectory expectation feature-matching algorithm is proposed that uses maximum entropy inverse reinforcement learning theory to learn the extracted expert-demonstrated trajectories and achieve automatic acquisition of the optimization function of the expert-demonstrated trajectory. Furthermore, a mapping model is constructed by combining the key driving scenario information that affects vehicle obstacle avoidance with the weight of the optimization function, and an anthropomorphic obstacle avoidance trajectory planning strategy for adaptive driving scenarios is proposed. Finally, the proposed strategy is verified based on real driving scenarios. The results show that the strategy can adjust the weight distribution of the trajectory optimization function in real time according to the “emergency degree” of obstacle avoidance and the state of the vehicle. Moreover, this strategy can generate anthropomorphic trajectories that are similar to expert-demonstrated trajectories, effectively improving the adaptability and acceptability of trajectories in driving scenarios.

Graphical abstract

Keywords

Obstacle avoidance trajectory planning / Inverse reinforcement theory / Anthropomorphic / Adaptive driving scenarios

Cite this article

EndNote

Ris (Procite)

Bibtex

Download citation ▾

Jian Wu, Yang Yan, Yulong Liu, Yahui Liu. Research on Anthropomorphic Obstacle Avoidance Trajectory Planning for Adaptive Driving Scenarios Based on Inverse Reinforcement Learning Theory. Engineering, 2024, 33(2): 133‒145 https://doi.org/10.1016/j.eng.2023.07.018

References

Publishing order | Descend order by publishing year | Descend order by cited within

[1]	C. Gao, G. Wang, W. Shi, Z. Wang, Y. Chen. Autonomous driving security: state of the art and challenges. IEEE Internet Things J, 9 (10) (2022), pp. 7572-7579

[2]	A. Benloucif, A.T. Nguyen, C. Sentouh, J.C. Popieul. Cooperative trajectory planning for haptic shared control between driver and automation in highway driving. IEEE Trans Ind Electron, 66 (12) (2019), pp. 9846-9857

[3]	D. Dolgov, S. Thrun, M. Montemerlo, J. Diebel. Practical search techniques in path planning for autonomous driving. Ann Arbor, 1001 (48105) (2009), pp. 18-80

[4]	Islam F, Narayanan V, Likhachev M. Dynamic multi-heuristic A. In:Proceedings of IEEE International Conference on Robotics and Automation (ICRA); 2015 May 26-30; Seattle, WA, USA. IEEE; 2015. p. 2376-82.

[5]	Kushleyev A, Likhachev M. Time-bounded lattice for efficient planning in dynamic environments. In:Proceedings of IEEE International Conference on Robotics and Automation (ICRA); 2009 May 12-17; Kobe, Japan. IEEE; 2009. p. 1662-8.

[6]	Arslan O, Berntorp K, Tsiotras P. Sampling-based algorithms for optimal motion planning using closed-loop prediction. In:Proceedings of IEEE International Conference on Robotics and Automation (ICRA); 2017 May 29-Jun 3; Singapore. IEEE; 2017. p. 4991-6.

[7]	LaValle SM, Kuffner JJ. Randomized kinodynamic planning. In:Proceedings of IEEE International Conference on Robotics and Automation (ICRA); 1999 May 10-15; Detroit, MI, USA. IEEE; 1999. p. 473-9.

[8]	Zucker M, Kuffner J, Branicky M. Multipartite RRTs for rapid replanning in dynamic environments. In:Proceedings of IEEE International Conference on Robotics and Automation (ICRA); 2007 Apr 10-14; Rome, Italy. IEEE; 2007. p. 1603-9.

[9]	T. Berglund, A. Brodnik, H. Jonsson, M. Staffanson, I. Soderkvist. Planning smooth and obstacle-avoiding B-spline paths for autonomous mining vehicles. IEEE Trans Autom Sci Eng, 7 (1) (2009), pp. 167-172

[10]	J. Wu, J. Zhang, B. Nie, Y. Liu, X. He. Adaptive control of PMSM servo system for steering-by-wire system with disturbances observation. IEEE Trans Transp Electrification, 8 (2) (2022), pp. 2015-2028

[11]	Rastelli JP, Lattarulo R, Nashashibi F. Dynamic trajectory generation using continuous-curvature algorithms for door to door assistance vehicles. In:Proceedings of IEEE Intelligent Vehicles Symposium Proceedings (IV); 2014 Jun 8-11; Dearborn, MI, USA. IEEE; 2014. p. 510-5.

[12]	Gu T, Dolan JM. On-road motion planning for autonomous vehicles. In:Proceedings of International Conference on Intelligent Robotics and Applications (ICIRA); 2012 Oct 3-5; Montreal, QC, Canada; 2012. p. 588-97.

[13]	Lattarulo R, González L, Perez J. Real-time trajectory planning method based on n-order curve optimization. In:Proceedings of International Conference on System Theory, Control and Computing (ICSTCC); 2020 Oct 8-10; Sinaia, Romania. IEEE; 2020. p. 751-6.

[14]	W. Lim, S. Lee, M. Sunwoo, K. Jo. Hybrid trajectory planning for autonomous driving in on-road dynamic scenarios. IEEE Trans Intell Transp Syst, 22 (1) (2019), pp. 341-355

[15]	B. Gutjahr, L. Gröll, M. Werling. Lateral vehicle trajectory optimization using constrained linear time-varying MPC. IEEE Trans Intell Transp Syst, 18 (6) (2016), pp. 1586-1595

[16]	McNaughton M, Urmson C, Dolan JM, Lee JW. Motion planning for autonomous driving with a conformal spatiotemporal lattice. In:Proceedings of IEEE International Conference on Robotics and Automation (ICRA); 2011 May 9-13; Shanghai, China. IEEE; 2011. p. 4889-95.

[17]	S. Dixit, U. Montanaro, M. Dianati, D. Oxtoby, T. Mizutani, A. Mouzakitis, et al.. Trajectory planning for autonomous high-speed overtaking in structured environments using robust MPC. IEEE Trans Intell Transp Syst, 21 (6) (2019), pp. 2310-2323

[18]	Y. Luo, Y. Xiang, K. Cao, K. Li. A dynamic automated lane change maneuver based on vehicle-to-vehicle communication. Transport Res C Emer, 62 (2016), pp. 87-102

[19]	D. Yang, S. Zheng, C. Wen, P.J. Jin, B. Ran. A dynamic lane-changing trajectory planning model for automated vehicles. Transport Res C Emer, 95 (2018), pp. 228-247

[20]	J. Chen, W. Zhan, M. Tomizuka. Autonomous driving motion planning with constrained iterative LQR. IEEE Trans Intell Vehicles, 4 (2) (2019), pp. 244-254

[21]	Liu Y, Liu Y, Ji X, Sun L, Tomizuka M, He X. Learning from demonstration: situation-adaptive lane change trajectory planning for automated highway driving. In:Proceedings of IEEE International Conference on Mechatronics and Automation (ICMA); 2020 Oct 13-16; Beijing, China. IEEE; 2020. p. 376-382.

[22]	Ziegler J, Bender P, Dang T, Stiller C. Trajectory planning for Bertha—a local, continuous method. In:Proceedings of IEEE International Vehicles Symposium Proceedings (IVSP); 2014 Jun 8-11; Dearborn, MI, USA. IEEE; 2014. p. 450-7.

[23]	Sun L, Peng C, Zhan W, Tomizuka M. A fast integrated planning and control framework for autonomous driving via imitation learning. In: Dynamic Systems and Control Conference (DSCC); 2018 Sep 30-Oct 3; Atlanta, GA, USA; 2018.

[24]	Wang Y, Pan D, Liu Z, Feng R. Study on lane change trajectory planning considering of driver characteristics. SAE Technical Paper 2018;2018-01-1627.

[25]	B. Zhou, Y. Wang, G. Yu, X. Wu. A lane-change trajectory model from drivers’ vision view. Transport Res C, 85 (2017), pp. 609-627

[26]	He X, Xu D, Zhao H, Moze M, Aioun F, Guillemard F. A human-like trajectory planning method by learning from naturalistic driving data. In:Proceedings of IEEE Intelligent Vehicles Symposium (IV); 2018 Jun 26-30; Changshu, China. IEEE; 2018. p. 339-46.

[27]	J. Wu, Q. Kong, K. Yang, Y. Liu, D. Cao, Z. Li. Research on the steering torque control for intelligent vehicles co-driving with the penalty factor of human-machine intervention. IEEE Trans Syst Man Cybern, 53 (1) (2023), pp. 59-70

[28]	A.T. Nguyen, J. Rath, T.M. Guerra, R. Palhares, H. Zhang. Robust set-invariance based fuzzy output tracking control for vehicle autonomous driving under uncertain lateral forces and steering constraints. IEEE Trans Intell Transp Syst, 22 (9) (2020), pp. 5849-5860

[29]	C. Miyajima, Y. Nishiwaki, K. Ozawa, T. Wakita, K. Itou, K. Takeda, et al.. Driver modeling based on driving behavior and its evaluation in driver identification. Proc IEEE, 95 (2) (2007), pp. 427-437

[30]	L. Xu, J. Hu, H. Jiang, W. Meng. Establishing style-oriented driver models by imitating human driving behaviors. IEEE Trans Intell Transp Syst, 16 (5) (2015), pp. 2522-2530

[31]	Cai P, Sun Y, Chen Y, Liu M. Vision-based trajectory planning via imitation learning for autonomous vehicles. In:Proceedings of IEEE Intelligent Transportation Systems Conference (ITSC); 2019 Oct 27-30; Auckland, New Zealand. IEEE; 2019. p. 2736-42.

[32]	H. Li, C. Wu, D. Chu, L. Lu, K. Cheng. Combined trajectory planning and tracking for autonomous vehicle considering driving styles. IEEE Access, 9 (2021), pp. 9453-9463

[33]	Zhang C, Chu D, Lyu N, Wu C. Trajectory planning and tracking for autonomous vehicle considering human driver personality. In:Proceedings of Conference on Vehicle Control and Intelligence (CVCI); 2019 Sep 21-22; Hefei, China; 2019. p. 1-6.

[34]	Wu P, Cao Y, He Y, Li D. Vision-based robot path planning with deep learning. In:Proceedings of International Conference on Computer Vision Systems (ICVS); 2017 Jul 10-13; Shenzhen, China; 2017.

[35]	Lenz D, Diehl F, Le M, Knoll A. Deep neural networks for Markovian interactive scene prediction in highway scenarios. In:Proceedings of IEEE Intelligent Vehicles Symposium (IV); 2017 Jun 11-14; Los Angeles, CA, USA. IEEE; 2017. p. 685-92.

[36]	Vallon C, Ercan Z, Carvalho A, Borrelli F. A machine learning approach for personalized autonomous lane change initiation and control. In:Proceedings of IEEE Intelligent Vehicles Symposium (IV); 2017 Jun 11-14; Los Angeles, CA, USA. IEEE; 2017. p. 1590-5.

[37]

Krajewski

, Bock

, Kloeker

, Eckstein

. The HighD dataset: a drone dataset of naturalistic vehicle trajectories on German highways for validation of highly automated driving systems. In:Proceedings of International Conference on Intelligent Transportation Systems (ITSC); 2018 Nov 4-7; Maui, HI, USA. IEEE; 2018. p. 2118-25.

[38]	Kuderer M, Gulati S, Burgard W. Learning driving styles for autonomous vehicles from demonstration. In:Proceedings of IEEE International Conference on Robotics and Automation (ICRA); 2015 May 26-30; Seattle, WA, USA. IEEE; 2015. p. 2641-6.