access icon free Traffic network micro-simulation model and control algorithm based on approximate dynamic programming

This study presents the adaptive traffic signal control algorithm in a distributed traffic network system. The proposed algorithm is based on a micro-simulation model and a reinforcement learning method, namely approximate dynamic programming (ADP). By considering traffic environment in discrete time, the microscopic traffic dynamic model is built. In particular, the authors explore a vehicle-following model using cellular automata theory. This vehicle-following model theoretically contributes to traffic network loading environment in an accessible way. To make the network coordinated, tunable state with weights of queue length and vehicles on lane is considered. The intersection can share information with each other in this state representation and make a joint action for intersection coordination. Moreover, the traffic signal control algorithm based on ADP method performs quite well in different performance measures witnessed by simulations. By comparing with other control methods, experimental results present that the proposed algorithm could be a potential candidate in an application of traffic network control system.

Inspec keywords: cellular automata; adaptive control; intelligent transportation systems; learning (artificial intelligence); dynamic programming

Other keywords: vehicle-following model; cellular automata theory; approximate dynamic programming; distributed traffic network system; reinforcement learning method; traffic network loading environment; adaptive traffic signal control algorithm; microscopic traffic dynamic model; traffic network microsimulation model; ADP method

Subjects: Traffic engineering computing; Control engineering computing; Road-traffic system control; Knowledge engineering techniques; Optimisation techniques; Automata theory

References

    1. 1)
    2. 2)
    3. 3)
      • 19. El-Tantawy, S., Abdulhai, B.: ‘Multi-agent reinforcement learning for integrated network of adaptive traffic signal controllers (MARLIN-ATSC)’. Proc. of IEEE Conf. on Intelligent Transportation Systems, 2012, pp. 319326.
    4. 4)
      • 20. Kuyer, L., Whiteson, S., Bakker, B., et al: ‘Multiagent reinforcement learning for urban traffic control using coordination graphs’, in Walter, D., et al (Ed.): ‘Machine learning and knowledge discovery in databases’ (Springer, 2008), pp. 656671.
    5. 5)
      • 25. Li, T., Zhao, D.B., Yi, J.Q.: ‘Adaptive dynamic programming for multi-intersections traffic signal intelligent control’. Proc. of IEEE Conf. on Intelligent Transportation Systems, 2008, pp. 286291.
    6. 6)
    7. 7)
    8. 8)
      • 16. France, J., Ghorbani, A.A.: ‘A multiagent system for optimizing urban traffic’. Int. IEEE/WIC Conf. on Intelligent Agent Technology, 2003, pp. 411414.
    9. 9)
      • 17. Yang, Z.S., Chen, X., Tang, Y.S., et al: ‘Intelligent cooperation control of urban traffic networks’. Proc. of Int. Conf. on Machine Learning and Cybernetics, 2005, vol. 3, pp. 14821486.
    10. 10)
    11. 11)
      • 3. Nagel, K., Schreckenberg, M.: ‘A cellular automaton model for freeway traffic’, J. Phys. I, 1992, 2, (12), pp. 22212229.
    12. 12)
    13. 13)
      • 11. Barto, A.G.: ‘Reinforcement learning: an introduction’ (MIT Press, Cambridge, 1998).
    14. 14)
    15. 15)
      • 24. Le, T., Cai, C., Walsh, T.: ‘Adaptive signal–vehicle cooperative controlling system’. Proc. of IEEE Conf. on Intelligent Transportation Systems, 2011, pp. 236241.
    16. 16)
    17. 17)
      • 22. Powell, W.B.: ‘Approximate dynamic programming: solving the curses of dimensionality’ (John Wiley & Sons, 2007, 2nd edn.).
    18. 18)
      • 29. Lämmer, S., Helbing, D.: ‘Self-control of traffic lights and vehicle flows in urban road networks’, J. Stat. Mech., Theory Exp., 2008, 4, pp. 136.
    19. 19)
    20. 20)
      • 31. Kok, J.R., Vlassis, N.: ‘Collaborative multiagent reinforcement learning by payoff propagation’, J. Mach. Learn. Res., 2006, 7, pp. 17891828.
    21. 21)
      • 21. Medina, J.C., Benekohal, R.F.: ‘Traffic signal control using reinforcement learning and the max-plus algorithm as a coordinating strategy’. Proc. of IEEE Conf. on Intelligent Transportation Systems, 2012, pp. 596601.
    22. 22)
      • 10. Wiering, M., Vreeken, J., Van Veenen, J., et al: ‘Simulation and optimization of traffic in a city’. IEEE Intelligent Vehicles Symp., 2004, pp. 453458.
    23. 23)
      • 30. Yin, B., Dridi, M., El Moudni, A.: ‘Traffic control model and algorithm based on decomposition of MDP’. Proc. of Int. Conf. on Control, Decision and Information Technologies, 2014, pp. 225230.
    24. 24)
    25. 25)
    26. 26)
    27. 27)
      • 9. Wiering, M.: ‘Multi-agent reinforcement learning for traffic light control’. Int. Conf. on Machine Learning, 2000, pp. 11511158.
    28. 28)
    29. 29)
      • 32. Cools, S.B., Gershenson, C., D'Hooghe, B.: ‘Self-organizing traffic lights: a realistic simulation’, in Mikhail, P., et al (Ed.): ‘Advances in applied self-organizing systems’ (Springer, 2008), pp. 4150.
    30. 30)
    31. 31)
      • 6. Abouaissa, H., Fliess, M., Join, C.: ‘Fast parametric estimation for macroscopic traffic flow model’. 17th IFAC World Congress, Seoul, South Korea, 2008.
    32. 32)
http://iet.metastore.ingenta.com/content/journals/10.1049/iet-its.2015.0108
Loading

Related content

content/journals/10.1049/iet-its.2015.0108
pub_keyword,iet_inspecKeyword,pub_concept
6
6
Loading