access icon free Distributed predictive cruise control based on reinforcement learning and validation on microscopic traffic simulation

This study proposes a novel distributed predictive cruise control (PCC) algorithm based on reinforcement learning. The algorithm aims at reducing idle time and maintaining an adjustable speed depending on the traffic signals. The effectiveness of the proposed approach has been validated through Paramics microscopic traffic simulations by proposing a scenario in Statesboro, Georgia. For different traffic demands, the travel time and fuel consumption rate of vehicles are compared between non-PCC and PCC algorithms. Microscopic traffic simulation results demonstrate that the proposed PCC algorithm will reduce the fuel consumption rate by 4.24% and decrease the average travel time by 3.78%.

Inspec keywords: control engineering computing; predictive control; velocity control; traffic engineering computing; distributed control; road traffic control; road vehicles; learning (artificial intelligence)

Other keywords: predictive cruise control algorithm; adjustable speed; traffic signals; fuel consumption rate; microscopic traffic simulation results; Paramics microscopic traffic simulations; nonPCC; PCC algorithm; reinforcement learning; average travel time; distributed predictive cruise control; traffic demands

Subjects: Multivariable control systems; Control engineering computing; Velocity, acceleration and rotation control; Traffic engineering computing; Road-traffic system control; Knowledge engineering techniques; Optimal control

References

    1. 1)
      • 11. Wei, K., Guo, S., Xu, K.: ‘CACC: a context-aware congestion control approach in smartphone networks’, IEEE Commun. Mag., 2014, 52, (6), pp. 4248.
    2. 2)
      • 13. Ploeg, J., Shukla, D.P., van de Wouw, N., et al: ‘Controller synthesis for string stability of vehicle platoons’, IEEE Trans. Intell. Transp. Syst., 2013, 15, (2), pp. 854865.
    3. 3)
      • 33. Wang, Z., Qin, Y., Hu, C., et al: ‘Fuzzy observer-based prescribed performance control of vehicle roll behavior via controllable damper’, IEEE Access, 2019, 7, pp. 19 47119 487.
    4. 4)
      • 19. Asadi, B., Vahidi, A.: ‘Predictive cruise control: utilizing upcoming traffic signal information for improving fuel economy and reducing trip time’, IEEE Trans. Control Syst. Technol., 2010, 19, (3), pp. 707714.
    5. 5)
      • 12. Zhu, Y., Zhao, D., Zhong, Z.: ‘Adaptive optimal control of heterogeneous CACC system with uncertain dynamics’, IEEE Trans. Control Syst. Technol., 2018, 27, (4), pp. 17721779.
    6. 6)
      • 27. Grigorescu, S.M., Trasnea, B., Marina, L., et al: ‘Neurotrajectory: a neuroevolutionary approach to local state trajectory learning for autonomous vehicles’, IEEE Robot. Automation Lett., 2019, 4, (4), pp. 34413448.
    7. 7)
      • 37. Jiang, Y., Jiang, Z.-P.: ‘Computational adaptive optimal control for continuous time linear systems with completely unknown dynamics’, Automatica, 2012, 48, (10), pp. 26992704.
    8. 8)
      • 30. El-Tantawy, S., Abdulhai, B., Abdelgawad, H.: ‘Multiagent reinforcement learning for integrated network of adaptive traffic signal controllers (MARLINATSC): methodology and large-scale application on downtown Toronto’, IEEE Trans. Intell. Transp. Syst., 2013, 14, (3), pp. 11401150.
    9. 9)
      • 4. Wang, F.-Y.: ‘Driving into the future with its’, IEEE Intell. Syst., 2006, 21, (3), pp. 9495.
    10. 10)
      • 28. Liu, T., Yu, H., Guo, H., et al: ‘Online energy management for multimode plug-in hybrid electric vehicles’, IEEE Trans. Ind. Inf., 2019, 15, (7), pp. 43524361.
    11. 11)
      • 6. Li, L., Zhang, H., Wang, X., et al: ‘Urban transit coordination using an artificial transportation system’, IEEE Trans. Intell. Transp. Syst., 2010, 12, (2), pp. 374383.
    12. 12)
      • 24. Gao, W., Jiang, Z.-P.: ‘Adaptive dynamic programming and adaptive optimal output regulation of linear systems’, IEEE Trans. Autom. Control, 2016, 61, (12), pp. 41644169.
    13. 13)
      • 15. Gao, W., Jiang, Z.P.: ‘Nonlinear and adaptive suboptimal control of connected vehicles: A global adaptive dynamic programming approach’, J Intell. Robot. Syst., 2017, 85, (3), pp. 597611.
    14. 14)
      • 29. Liu, T., Hu, X., Hu, W., et al: ‘A heuristic planning reinforcement learning-based energy management for power-split plug-in hybrid electric vehicles’, IEEE Trans. Ind. Inf., 2019, 15, (12), pp. 64366445.
    15. 15)
      • 20. Park, S., Rakha, H., Ahn, K., et al: ‘Predictive eco-cruise control: algorithm and potential benefits’. 2011 IEEE Forum on Integrated and Sustainable Transportation Systems, Vienna, Austria, 2011, pp. 394399.
    16. 16)
      • 35. Li, S., Li, K., Rajamani, R., et al: ‘Model predictive multi-objective vehicular adaptive cruise control’, IEEE Trans. Control Syst. Technol., 2010, 19, (3), pp. 556566.
    17. 17)
      • 25. Li, N., Oyler, D.W., Zhang, M., et al: ‘Game theoretic modeling of driver and vehicle interactions for verification and validation of autonomous vehicle control systems’, IEEE Trans. Control Syst. Technol., 2018, 26, (5), pp. 17821797.
    18. 18)
      • 16. Delis, A.I., Nikolos, I.K., Papageorgiou, M.: ‘Simulation of the penetration rate effects of ACC and CACC on macroscopic traffic dynamics’. 2016 IEEE 19th Int. Conf. on Intelligent Transportation Systems (ITSC), Rio de Janeiro, Brazil, 2016, pp. 336341.
    19. 19)
      • 5. Wang, J., Sun, Y., Liu, Z., et al: ‘Route planning based on floyd algorithm for intelligence transportation system’, 2007 IEEE Int. Conf. on Integration Technology, Shenzhen, China, 2007, pp. 544546.
    20. 20)
      • 39. Wong, J.Y.: ‘Theory of ground vehicles’ (john wiley&sons’, Inc., New York, 2001).
    21. 21)
      • 3. Li, L., Li, X., Cheng, C., et al: ‘Research collaboration and its topic evolution: 10 years at t-its’, IEEE Trans. Intell. Transp. Syst., 2010, 11, (3), pp. 517523.
    22. 22)
      • 8. Lidstrom, K., Sjoberg, K., Holmberg, U., et al: ‘A modular CACC system integration and design’, IEEE Trans. Intell. Transp. Syst., 2012, 13, (3), pp. 10501061.
    23. 23)
      • 7. INRIX: ‘Congestion Costs Each American 97 h, 1,348 A Year’, Tech. Rep., 2019. [Online]. Available: http://inrix.com/press-releases/scorecard-2018-us/.
    24. 24)
      • 9. Öncü, S., Ploeg, J., Van de Wouw, N., et al: ‘Cooperative adaptive cruise control: network-aware analysis of string stability’, IEEE Trans. Intell. Transp. Syst., 2014, 15, (4), pp. 15271537.
    25. 25)
      • 23. Lv, Y., Ren, X., Na, J.: ‘Online optimal solutions for multi-player nonzero-sum game with completely unknown dynamics’, Neurocomput., 2018, 283, pp. 8797.
    26. 26)
      • 17. Ngoduy, D.: ‘Application of gas-kinetic theory to modelling mixed traffic of manual and ACC vehicles’, Transportmetrica, 2012, 8, (1), pp. 4360.
    27. 27)
      • 40. Rakha, H., Lucic, I., Demarchi, S.H., et al: ‘Vehicle dynamics model for predicting maximum truck acceleration levels’, J. Transp. Eng., 2001, 127, (5), pp. 418425.
    28. 28)
      • 22. Lv, Y., Na, J., Ren, X.: ‘Online H-infinity control for completely unknown nonlinear systems via an identifier–critic-based ADP structure’, Int. J. Control, 2019, 92, (1), pp. 100111.
    29. 29)
      • 36. Lewis, F., Vrabie, D., Syrmos, V.: ‘Reinforcement learning and optimal adaptive control’, in Ojptimal Control (John Wiley & Sons, Inc., Hoboken, NJ, USA, 2012, Third Edition), pp. 461513.
    30. 30)
      • 18. Bloomberg, L., Dale, J.: ‘Comparison of VISSIM and CORSIM traffic simulation models on a congested network’, Transp. Res. Rec., 2000, 1727, (1), pp. 5260.
    31. 31)
      • 2. Li, L., Li, X., Li, Z., et al: ‘A bibliographic analysis of the ieee transactions on intelligent transportation systems literature’, IEEE Trans. Intell. Transp. Syst., 2010, 11, (2), pp. 251255.
    32. 32)
      • 31. Liu, T., Wang, B., Yang, C.: ‘Online Markov chain-based energy management for a hybrid tracked vehicle with speedy q-learning’, Energy, 2018, 160, pp. 544555.
    33. 33)
      • 34. Ioannou, P.A., Chien, C.-C.: ‘Autonomous intelligent cruise control’, IEEE Trans. Veh. Technol., 1993, 42, (4), pp. 657672.
    34. 34)
      • 10. Guo, G., Yue, W.: ‘Sampled-data cooperative adaptive cruise control of vehicles with sensor failures’, IEEE Trans. Intell. Transp. Syst., 2014, 15, (6), pp. 24042418.
    35. 35)
      • 26. Huang, Z., Xu, X., He, H., et al: ‘Parameterized batch reinforcement learning for longitudinal control of autonomous land vehicles’, IEEE Trans. Syst., Man, Cybernetics, Syst., 2019, 49, (4), pp. 730741.
    36. 36)
      • 32. Li, Z., Liu, P., Xu, C., et al: ‘Reinforcement learning-based variable speed limit control strategy to reduce traffic congestion at freeway recurrent bottlenecks’, IEEE Trans. Intell. Transp. Syst., 2017, 18, (11), pp. 32043217.
    37. 37)
      • 14. Gao, W., Gao, J., Ozbay, K., et al: ‘Reinforcement-learning-based cooperative adaptive cruise control of buses in the Lincoln tunnel corridor with time-varying topology’, IEEE Trans. Intell. Transp. Syst., 2019, 20, pp. 37963805.
    38. 38)
      • 21. Gao, W., Odekunle, A., Chen, Y., et al: ‘Predictive cruise control of connected and autonomous vehicles via reinforcement learning’, IET Control Theory Appl., 2019, 13, (17), pp. 28492855.
    39. 39)
      • 38. Rakha, H.A., Ahn, K., Moran, K., et al: ‘Virginia tech comprehensive power-based fuel consumption model: model development and testing’, Transp. Res. D, Transp. Environ., 2011, 16, (7), pp. 492503.
    40. 40)
      • 1. Zhu, F., Li, Z., Chen, S., et al: ‘Parallel transportation management and control system and its applications in building smart cities’, IEEE Trans. Intell. Transp. Syst., 2016, 17, (6), pp. 15761585.
http://iet.metastore.ingenta.com/content/journals/10.1049/iet-its.2019.0404
Loading

Related content

content/journals/10.1049/iet-its.2019.0404
pub_keyword,iet_inspecKeyword,pub_concept
6
6
Loading