Traffic network micro-simulation model and control algorithm based on approximate dynamic programming

Traffic network micro-simulation model and control algorithm based on approximate dynamic programming

For access to this article, please select a purchase option:

Buy article PDF
(plus tax if applicable)
Buy Knowledge Pack
10 articles for $120.00
(plus taxes if applicable)

IET members benefit from discounts to all IET publications and free access to E&T Magazine. If you are an IET member, log in to your account and the discounts will automatically be applied.

Learn more about IET membership 

Recommend Title Publication to library

You must fill out fields marked with: *

Librarian details
Your details
Why are you recommending this title?
Select reason:
IET Intelligent Transport Systems — Recommend this title to your library

Thank you

Your recommendation has been sent to your librarian.

This study presents the adaptive traffic signal control algorithm in a distributed traffic network system. The proposed algorithm is based on a micro-simulation model and a reinforcement learning method, namely approximate dynamic programming (ADP). By considering traffic environment in discrete time, the microscopic traffic dynamic model is built. In particular, the authors explore a vehicle-following model using cellular automata theory. This vehicle-following model theoretically contributes to traffic network loading environment in an accessible way. To make the network coordinated, tunable state with weights of queue length and vehicles on lane is considered. The intersection can share information with each other in this state representation and make a joint action for intersection coordination. Moreover, the traffic signal control algorithm based on ADP method performs quite well in different performance measures witnessed by simulations. By comparing with other control methods, experimental results present that the proposed algorithm could be a potential candidate in an application of traffic network control system.


    1. 1)
    2. 2)
    3. 3)
      • 3. Nagel, K., Schreckenberg, M.: ‘A cellular automaton model for freeway traffic’, J. Phys. I, 1992, 2, (12), pp. 22212229.
    4. 4)
    5. 5)
    6. 6)
      • 6. Abouaissa, H., Fliess, M., Join, C.: ‘Fast parametric estimation for macroscopic traffic flow model’. 17th IFAC World Congress, Seoul, South Korea, 2008.
    7. 7)
    8. 8)
    9. 9)
      • 9. Wiering, M.: ‘Multi-agent reinforcement learning for traffic light control’. Int. Conf. on Machine Learning, 2000, pp. 11511158.
    10. 10)
      • 10. Wiering, M., Vreeken, J., Van Veenen, J., et al: ‘Simulation and optimization of traffic in a city’. IEEE Intelligent Vehicles Symp., 2004, pp. 453458.
    11. 11)
      • 11. Barto, A.G.: ‘Reinforcement learning: an introduction’ (MIT Press, Cambridge, 1998).
    12. 12)
    13. 13)
    14. 14)
    15. 15)
    16. 16)
      • 16. France, J., Ghorbani, A.A.: ‘A multiagent system for optimizing urban traffic’. Int. IEEE/WIC Conf. on Intelligent Agent Technology, 2003, pp. 411414.
    17. 17)
      • 17. Yang, Z.S., Chen, X., Tang, Y.S., et al: ‘Intelligent cooperation control of urban traffic networks’. Proc. of Int. Conf. on Machine Learning and Cybernetics, 2005, vol. 3, pp. 14821486.
    18. 18)
    19. 19)
      • 19. El-Tantawy, S., Abdulhai, B.: ‘Multi-agent reinforcement learning for integrated network of adaptive traffic signal controllers (MARLIN-ATSC)’. Proc. of IEEE Conf. on Intelligent Transportation Systems, 2012, pp. 319326.
    20. 20)
      • 20. Kuyer, L., Whiteson, S., Bakker, B., et al: ‘Multiagent reinforcement learning for urban traffic control using coordination graphs’, in Walter, D., et al (Ed.): ‘Machine learning and knowledge discovery in databases’ (Springer, 2008), pp. 656671.
    21. 21)
      • 21. Medina, J.C., Benekohal, R.F.: ‘Traffic signal control using reinforcement learning and the max-plus algorithm as a coordinating strategy’. Proc. of IEEE Conf. on Intelligent Transportation Systems, 2012, pp. 596601.
    22. 22)
      • 22. Powell, W.B.: ‘Approximate dynamic programming: solving the curses of dimensionality’ (John Wiley & Sons, 2007, 2nd edn.).
    23. 23)
    24. 24)
      • 24. Le, T., Cai, C., Walsh, T.: ‘Adaptive signal–vehicle cooperative controlling system’. Proc. of IEEE Conf. on Intelligent Transportation Systems, 2011, pp. 236241.
    25. 25)
      • 25. Li, T., Zhao, D.B., Yi, J.Q.: ‘Adaptive dynamic programming for multi-intersections traffic signal intelligent control’. Proc. of IEEE Conf. on Intelligent Transportation Systems, 2008, pp. 286291.
    26. 26)
    27. 27)
    28. 28)
    29. 29)
      • 29. Lämmer, S., Helbing, D.: ‘Self-control of traffic lights and vehicle flows in urban road networks’, J. Stat. Mech., Theory Exp., 2008, 4, pp. 136.
    30. 30)
      • 30. Yin, B., Dridi, M., El Moudni, A.: ‘Traffic control model and algorithm based on decomposition of MDP’. Proc. of Int. Conf. on Control, Decision and Information Technologies, 2014, pp. 225230.
    31. 31)
      • 31. Kok, J.R., Vlassis, N.: ‘Collaborative multiagent reinforcement learning by payoff propagation’, J. Mach. Learn. Res., 2006, 7, pp. 17891828.
    32. 32)
      • 32. Cools, S.B., Gershenson, C., D'Hooghe, B.: ‘Self-organizing traffic lights: a realistic simulation’, in Mikhail, P., et al (Ed.): ‘Advances in applied self-organizing systems’ (Springer, 2008), pp. 4150.

Related content

This is a required field
Please enter a valid email address