Forward search algorithm based on dynamic programming for real-time adaptive traffic signal control

Forward search algorithm based on dynamic programming for real-time adaptive traffic signal control

For access to this article, please select a purchase option:

Buy article PDF
(plus tax if applicable)
Buy Knowledge Pack
10 articles for £75.00
(plus taxes if applicable)

IET members benefit from discounts to all IET publications and free access to E&T Magazine. If you are an IET member, log in to your account and the discounts will automatically be applied.

Learn more about IET membership 

Recommend Title Publication to library

You must fill out fields marked with: *

Librarian details
Your details
Why are you recommending this title?
Select reason:
IET Intelligent Transport Systems — Recommend this title to your library

Thank you

Your recommendation has been sent to your librarian.

The scheduling of traffic signal at intersections is involved in an application of artificial intelligence system. This study presents a new forward search algorithm based on dynamic programming (FSDP) under a decision tree, and explores an efficient solution for real-time adaptive traffic signal control policy. Traffic signal control with cases of fixed phase sequence and variable phase sequence are both considered in the algorithm. Owing to the properties of forward research dynamic programming and the process optimisation of repeated or invalid traffic states the authors proposed, FSDP algorithm reduces the number of states and saves much computation time. Consequently, FSDP is certain to be an on-line algorithm through its application to a complicated traffic control problem. Moreover, the labelled position method is firstly proposed in the author's study to search the optimal policy after reaching the goal state. For practical operations, this new algorithm is extended by adding the rolling horizon approach, and some derived methods are compared with the optimal fixed-time control and adaptive control on the evaluation of traffic delay. Experimental results obtained by the simulations of symmetrical and asymmetrical traffic flow scenarios show that the FSDP method can perform quite well with high efficiency and good qualities in traffic control.


    1. 1)
      • 1. Cheng, D., Messer, C.J., Tian, Z.Z., Liu, J.: ‘Modification of Webster's minimum delay cycle length equation based on hcm 2000’. 81th Ann. Meeting of the Transportation Research Board, Washington, DC, 2003.
    2. 2)
    3. 3)
    4. 4)
    5. 5)
    6. 6)
    7. 7)
    8. 8)
      • 8. Kuyer, L., Whiteson, S., Bakker, B., Vlassis, N.: ‘Multiagent reinforcement learning for urban traffic control using coordination graphs’. in Daelemans, W., Morik, K. (Eds.): ‘Machine learning and knowledge discovery in databases’, (Springer, 2008), pp. 656671.
    9. 9)
    10. 10)
      • 10. Gartner, N.H.: ‘Opac: A demand-responsive strategy for traffic signal control’, Transp. Res. Rec., 1983, 906, pp. 7581.
    11. 11)
      • 11. Gartner, N.H., Pooran, F.J., Andrews, C.M.: ‘Implementation of the opac adaptive control strategy in a traffic signal network’. Proc. IEEE Int. Conf. Intelligent Transportation Systems, 2001, pp. 195200.
    12. 12)
      • 12. Henry, J.J., Farges, J.L., Tuffal, J.: ‘The prodyn real time traffic algorithm’. IFAC-IFIP-IFORS Conf. on Control in Transportation System, 1984.
    13. 13)
      • 13. Head, K.L., Mirchandani, P.B., Sheppard, D.: ‘Hierarchical framework for real-time traffic control’, Transp. Res. Rec., 1992, 1360, pp. 8288.
    14. 14)
    15. 15)
      • 15. Busoniu, L., Babuska, R., De Schutter, B., Ernst, D.: ‘Reinforcement learning and dynamic programming using function approximators’ (CRC Press, 2010).
    16. 16)
      • 16. Yin, B., Dridi, M., EL Moudni, A.: ‘Markov decision process for traffic control at an isolated intersection’. IEEE 25th Int. Conf. on Tools with Artificial Intelligence (ICTAI), 2013, pp. 789794.
    17. 17)
    18. 18)
      • 18. Gartner, N.H., Pooran, F.J., Andrews, C.M.: ‘Evaluation of optimized policies for adaptive control strategy’, Transp. Res. Rec., 1991, 1324, pp. 105114.
    19. 19)
    20. 20)
    21. 21)
      • 21. Si, J., Barto, A.G., Powell, W.B., Wunsch, D.C.: ‘Handbook of learning and approximate dynamic programming’ (IEEE Press, 2004).
    22. 22)
      • 22. Powell, W.B.: ‘Approximate dynamic programming: solving the curses of dimensionality’ (John Wiley & Sons, 2007, 2nd edn.), p. 703.
    23. 23)
    24. 24)
    25. 25)
      • 25. Li, T., Zhao, D.B., Yi, J.Q.: ‘Adaptive dynamic programming for multi-intersections traffic signal intelligent control’. 11th Int. IEEE Conf. on Intelligent Transportation Systems (ITSC), 2008, pp. 286291.
    26. 26)
      • 26. Yin, B., Dridi, M., EL Moudni, A.: ‘Approximate dynamic programming for traffic signal control at isolated intersection’. inSilhavy, R., Senkerik, R., Oplatkova, Z.K., Silhavy, P., Prokopova, Z. (Eds.): ‘Modern trends and techniques in computer science’ (Springer, 2014), pp. 369381.
    27. 27)
    28. 28)
    29. 29)
      • 29. Bonet, B., Geffner, H.: ‘Labeled rtdp: improving the convergence of real-time dynamic programming’. Int. Conf. on Automated Planning and Scheduling (ICAPS), 2003, vol. 3, pp. 1221.
    30. 30)
    31. 31)
      • 31. McMahan, H.B., Likhachev, M., Gordon, G.J.: ‘Bounded real-time dynamic programming: Rtdp with monotone upper bounds and performance guarantees’. Proc. 22th Int. Conf. on Machine learning, 2005, pp. 569576.
    32. 32)
      • 32. Smith, T., Simmons, R.: ‘Focused real-time dynamic programming for mdps: squeezing more out of a heuristic’. Proc. Natl. Conf. on Artificial Intelligence, 2006, vol. 21, no. 2, pp. 12271232.

Related content

This is a required field
Please enter a valid email address