Your browser does not support JavaScript!
http://iet.metastore.ingenta.com
1887

access icon free Predictive cruise control of connected and autonomous vehicles via reinforcement learning

  • Author(s): Weinan Gao 1 ; Adedapo Odekunle 1 ; Yunfeng Chen 2 ; Zhong-Ping Jiang 3
    • View affiliations
  • Source: Volume 13, Issue 17, 26 November 2019, p. 2849 – 2855
    DOI:  10.1049/iet-cta.2018.6031 , Print ISSN 1751-8644, Online ISSN 1751-8652
© The Institution of Engineering and Technology
Received 07/09/2018, Accepted 29/11/2018, Revised 26/10/2018, Published 02/12/2018

Predictive cruise control concerns designing controllers for autonomous vehicles using the broadcasted information from the traffic lights such that the idle time around the intersection can be reduced. This study proposes a novel adaptive optimal control approach based on reinforcement learning to solve the predictive cruise control problem of a platoon of connected and autonomous vehicles. First, the reference velocity is determined for each autonomous vehicle in the platoon. Second, a data-driven adaptive optimal control algorithm is developed to estimate the gains of the desired distributed optimal controllers without the exact knowledge of system dynamics. The obtained controller is able to regulate the headway, velocity, and acceleration of each vehicle in a suboptimal sense. The goal of trip time reduction is achieved without compromising vehicle safety and passenger comfort. Numerical simulations are presented to validate the efficacy of the proposed methodology.

Inspec keywords: adaptive control; optimal control; predictive control; distributed control; mobile robots; acceleration control; learning (artificial intelligence); road vehicles; road traffic control; velocity control

Other keywords: connected vehicles; vehicle safety; reinforcement learning; headway regulation; velocity regulation; data-driven adaptive optimal control algorithm; suboptimal control; acceleration regulation; autonomous vehicle; passenger comfort; predictive cruise control; distributed optimal controllers

Subjects: Multivariable control systems; Self-adjusting control systems; Optimal control; Learning in AI (theory); Road-traffic system control; Velocity, acceleration and rotation control; Mobile robots

References

    1. 1)
      • 26. Meng, Z., Yang, T., Dimarogonas, D.V., et al: ‘Coordinated output regulation of heterogeneous linear systems under switching topologies’, Automatica, 2015, 53, pp. 362–368.
    2. 2)
      • 13. Gao, W., Jiang, Z.P., Ozbay, K., et al: ‘Data-driven cooperative adaptive cruise control of buses on the exclusive bus lane of the lincoln tunnel corridor’. TRB Annual Meeting, Washington, DC, 2018.
    3. 3)
      • 16. Kavurucu, Y., Ensar, T.: ‘Predictive cruise control’. Electric Electronics, Computer Science, Biomedical Engineerings Meeting, Istanbul, Turkey, 2017, pp. 1–4.
    4. 4)
      • 6. Oncu, S., Ploeg, J., van de Wouw, N., , et al: ‘Cooperative adaptive cruise control: network-aware analysis of string stability’, IEEE Trans. Intell. Transp. Syst., 2014, 15, (4), pp. 1527–1537.
    5. 5)
      • 24. Jiang, Y., Jiang, Z.P.: ‘Robust adaptive dynamic programming’ (Wiley-IEEE Press, Hoboken, NJ, 2017).
    6. 6)
      • 18. Fang, H., Wu, D., Yang, T.: ‘Cooperative management of a lithium-ion battery energy storage network: a distributed MPC approach’. 2016 IEEE 55th Conf. on Decision and Control (CDC), Las Vegas, NV, USA, December 2016, pp. 4226–4232.
    7. 7)
      • 11. van Arem, B., van Driel, C., Visser, R.: ‘The impact of cooperative adaptive cruise control on traffic-flow characteristics’, IEEE Trans. Intell. Transp. Syst., 2006, 7, (4), pp. 429–436.
    8. 8)
      • 19. Sutton, R.S., Barto, A.G.: ‘Introduction to reinforcement learning’ (MIT Press, Cambridge, MA, 1998).
    9. 9)
      • 23. Jiang, Y., Fan, J., Chai, T., et al: ‘Data-driven flotation industrial process operational optimal control based on reinforcement learning’, IEEE Trans. Ind. Inf., 2017, 14, (5), pp. 1974–1989.
    10. 10)
      • 12. Shladover, S., Su, D., Lu, X.-Y.: ‘Impacts of cooperative adaptive cruise control on freeway traffic flow’, Transp. Res. Record, 2012, 2324, pp. 63–70.
    11. 11)
      • 8. Guo, G., Yue, W.: ‘Sampled-data cooperative adaptive cruise control of vehicles with sensor failures’, IEEE Trans. Intell. Transp. Syst., 2014, 15, (6), pp. 2404–2418.
    12. 12)
      • 1. ‘The intelligent transportation systems for traffic signal control deployment benefits and lessons learned’. Tech. Rep., US Department of Transportation, Washington, DC, 2007.
    13. 13)
      • 22. Wang, D., Liu, D., Li, H., et al: ‘An approximate optimal control approach for robust stabilization of a class of discrete-time nonlinear systems with uncertainties’, IEEE Trans. Syst. Man Cybern. Syst., 2016, 46, (5), pp. 713–717.
    14. 14)
      • 14. Lee, J., Park, B.: ‘Development and evaluation of a cooperative vehicle intersection control algorithm under the connected vehicles environment’, IEEE Trans. Intell. Transp. Syst., 2012, 13, (1), pp. 81–90.
    15. 15)
      • 15. Asadi, B., Vahidi, A.: ‘Predictive cruise control: utilizing upcoming traffic signal information for improving fuel economy and reducing trip time’, IEEE Trans. Control Syst. Technol., 2011, 19, (3), pp. 707–714.
    16. 16)
      • 30. Gao, W., Jiang, Z.P., Lewis, F.L., et al: ‘Leader-to-formation stability of multi-agent systems: an adaptive optimal control approach’, IEEE Trans. Autom. Control, 2017, 63, (10), pp. 3581–3587.
    17. 17)
      • 7. Desjardins, C., Chaib-draa, B.: ‘Cooperative adaptive cruise control: a reinforcement learning approach’, IEEE Trans. Intell. Transp. Syst., 2011, 12, (4), pp. 1248–1260.
    18. 18)
      • 5. Gao, W., Jiang, Z.P., Ozbay, K.: ‘Data-driven adaptive optimal control of connected vehicles’, IEEE Trans. Intell. Transp. Syst., 2017, 18, (5), pp. 1122–1133.
    19. 19)
      • 21. Fan, Q.Y., Yang, G.H.: ‘Adaptive actor-critic design-based integral sliding-mode control for partially unknown nonlinear systems with input disturbances’, IEEE Trans. Neural Netw. Learn. Syst., 2016, 27, (1), pp. 165–177.
    20. 20)
      • 31. Lewis, F.L., Vamvoudakis, K.G.: ‘Reinforcement learning for partially observable dynamic processes: adaptive dynamic programming using measured output data’, IEEE Trans. Syst. Man Cybern., B, Cybern., 2011, 41, (1), pp. 14–25.
    21. 21)
      • 17. Alrifaee, B., Jodar, J.G., Abel, D.: ‘Predictive cruise control for energy saving in REEV using V2I information’. Mediterranean Conf. on Control and Automation, Torremolinos, Spain, 2015, pp. 82–87.
    22. 22)
      • 10. Yang, T., Wan, Y., Wang, H., et al: ‘Global optimal consensus for discrete-time multi-agent systems with bounded controls’, Automatica, 2018, 97, pp. 182–185.
    23. 23)
      • 3. Zhang, Y.J., Malikopoulos, A.A., Cassandras, C.G.: ‘Optimal control and coordination of connected and automated vehicles at urban traffic intersections’. Proc. American Control Conf., Boston, MA, USA, 2016, pp. 6227–6232.
    24. 24)
      • 29. Kleinman, D.: ‘On an iterative technique for Riccati equation computations’, IEEE Trans. Autom. Control, 1968, 13, (1), pp. 114–115.
    25. 25)
      • 20. Vamvoudakis, K.G.: ‘Event-triggered optimal adaptive control algorithm for continuous-time nonlinear systems’, IEEE/CAA J. Autom. Sin., 2014, 3, pp. 282–293.
    26. 26)
      • 4. Talebpour, A., Mahmassani, H.S.: ‘Influence of connected and autonomous vehicles on traffic flow stability and throughput’, Transp. Res. C, Emerg. Technol., 2016, 71, pp. 143–163.
    27. 27)
      • 9. Yang, T., Meng, Z., Dimarogonas, D.V., et al: ‘Global consensus for discrete-time multi-agent systems with input saturation constraints’, Automatica, 2014, 50, (2), pp. 499–506.
    28. 28)
      • 25. Gao, W., Jiang, Z.P.: ‘Adaptive dynamic programming and adaptive optimal output regulation of linear systems’, IEEE Trans. Autom. Control, 2016, 61, (12), pp. 4164–4169.
    29. 29)
      • 28. Stankovic, S.S., Stanojevic, M.J., Siljak, D.D.: ‘Decentralized overlapping control of a platoon of vehicles’, IEEE Trans. Control Syst. Technol., 2000, 8, (5), pp. 816–832.
    30. 30)
      • 27. Li, S., Li, K., Rajamani, R., et al: ‘Model predictive multi-objective vehicular adaptive cruise control’, IEEE Trans. Control Syst. Technol., 2011, 19, (3), pp. 556–566.
    31. 31)
      • 2. Sun, X., Chen, X., Qi, Y., et al: ‘Analyzing the effects of different advanced traffic signal status warning systems on vehicle emission reductions at signalized intersections’. Transporation Research Board Annual Meeting, Washington, DC, 2016.
    32. 32)
      • 32. Gao, W., Jiang, Z.P.: ‘Learning-based adaptive optimal tracking control of strict-feedback nonlinear systems’, IEEE Trans. Neural Netw. Learn. Syst., 2018, 29, (6), pp. 2614–2624.
http://iet.metastore.ingenta.com/content/journals/10.1049/iet-cta.2018.6031
Loading

Related content

content/journals/10.1049/iet-cta.2018.6031
pub_keyword,iet_inspecKeyword,pub_concept
6
6
Loading
Print this page
This is a required field
Please enter a valid email address