Urban traffic signal control using reinforcement learning agents

Urban traffic signal control using reinforcement learning agents

For access to this article, please select a purchase option:

Buy article PDF
(plus tax if applicable)
Buy Knowledge Pack
10 articles for $120.00
(plus taxes if applicable)

IET members benefit from discounts to all IET publications and free access to E&T Magazine. If you are an IET member, log in to your account and the discounts will automatically be applied.

Learn more about IET membership 

Recommend Title Publication to library

You must fill out fields marked with: *

Librarian details
Your details
Why are you recommending this title?
Select reason:
IET Intelligent Transport Systems — Recommend this title to your library

Thank you

Your recommendation has been sent to your librarian.

This study presents a distributed multi-agent-based traffic signal control for optimising green timing in an urban arterial road network to reduce the total travel time and delay experienced by vehicles. The proposed multi-agent architecture uses traffic data collected by sensors at each intersection, stored historical traffic patterns and data communicated from agents in adjacent intersections to compute green time for a phase. The parameters like weights, threshold values used in computing the green time is fine tuned by online reinforcement learning with an objective to reduce overall delay. PARAMICS software was used as a platform to simulate 29 signalised intersection at Central Business District of Singapore and test the performance of proposed multi-agent traffic signal control for different traffic scenarios. The proposed multi-agent reinforcement learning (RLA) signal control showed significant improvement in mean time delay and speed in comparison to other traffic control system like hierarchical multi-agent system (HMS), cooperative ensemble (CE) and actuated control.


    1. 1)
      • P. Koonce . (2008) Traffic signal timing manual.
    2. 2)
      • Sanchez, J.J., Galan, M., Rubio, E.: `Genetic algorithms and cellular automata: a new architecture for traffic light cycles optimization', Proc. Congress on Evolutionary Computation, 19–23 June 2004, 2004, Piscataway, NJ, USA, p. 1668–1674.
    3. 3)
      • Hoar, R., Penner, J., Jacob, C.: `Evolutionary swarm traffic: if ant roads had traffic lights', Proc. 2002 World Congress on Computational Intelligence – WCCI’02, 12–17 May 2002, 2002, Piscataway, NJ, USA, p. 1910–1915.
    4. 4)
      • Ishihara, H., Fukuda, T.: `Traffic signal networks simulator using emotional algorithm with individuality', Proc. IEEE Intelligent Transportation Systems, 25–29 August, 2001, Oakland, CA, USA, p. 1034–1039.
    5. 5)
    6. 6)
      • P.B. Hunt , D.I. Robertson , R.D. Bretherton , R.I. Winton . (1981) SCOOT – a traffic responsive method of coordinating signals.
    7. 7)
      • Peck, C., Gorton, P.T.W., Liren, D.: `Application of SCOOT in developing countries', Third Int. Conf. on Road Traffic Control, 1–3 May 1990, London, England, p. 104–109.
    8. 8)
      • A.G. Sims , K.W. Dobinson . The Sydney Coordinated Adaptive Traffic (SCAT) system philosophy and benefits. IEEE Trans. Veh. Technol. , 130 - 137
    9. 9)
      • Lowrie, P.R.: `The Sydney Coordinated Adaptive Traffic System-principles, methodology, algorithms', Int. Conf. on Road Traffic Signalling, 30 March–1 April 1982, London, UK, p. 67–70.
    10. 10)
      • C.K. Keong . The GLIDE system – Singapore's urban traffic control system. Transp. Rev., Transnatl. Transdiscipl. J. , 295 - 305
    11. 11)
    12. 12)
      • D.A. Roozemond . Using intelligent agents for pro-active, real-time urban intersection control. Eur. J. Oper. Res. , 293 - 301
    13. 13)
      • Mizuno, K., Nishihara, S.: `Distributed constraint satisfaction for urban traffic signal control', Second Int. Conf. on Knowledge Science, Engineering and Management. KSEM 2007, 28–30 November 2007, 2007, Berlin, Germany, p. 73–84.
    14. 14)
      • De Oliveira, D., Bazzan, A.L.C.: `Traffic lights control with adaptive group formation based on swarm intelligence', Ant Colony Optimization and Swarm Intelligence. Proc. Fifth Int. Workshop, ANTS 2006, 4–7 September 2006, Berlin, Germany, p. 520–521.
    15. 15)
      • Choy, M.C., Cheu, R.L., Srinivasan, D., Logi, F.: `Real-time coordinated signal control through use of agents with online reinforcement learning', Transportation Research Board Meeting (82nd), 2003, Washington, DC, p. 64–75.
    16. 16)
      • E. Camponogara , W. Kraus . Distributed learning agents in urban traffic control. Prog. Artif. Intell. , 324 - 335
    17. 17)
      • C. Watkins , P. Dayan . Technical note: Q-learning. Mach. Learn. , 279 - 292
    18. 18)
      • J.D.C. Little . A proof for the queuing formula: L={lambda} W. Oper. Res. , 383 - 387
    19. 19)
      • ‘Highway capacity manual – HCM2000’ (Transportation Research Board, National Research Council, 2000).
    20. 20)
      • Balaji, P.G., Srinivasan, D., Chen-Khong, T.: `Coordination in distributed multi-agent system using type-2 fuzzy decision systems', IEEE 16th Int. Conf. on Fuzzy Systems (FUZZ-IEEE), 1–6 June 2008, Piscataway, NJ, USA, p. 2291–2298.
    21. 21)
      • M.C. Choy , D. Srinivasan , R.L. Cheu . Neural networks for continuous online learning and control. IEEE Trans. Neural Netw. , 1511 - 1531
    22. 22)
      • D. Srinivasan , M. Choy . Distributed problem solving using evolutionary learning in multi-agent systems. Adv. Evol. Comput. Syst. Des. , 211 - 227
    23. 23)
      • M.C. Choy , D. Srinivasan , R.L. Cheu . Cooperative, hybrid agent architecture for real-time traffic signal control. IEEE Trans. Syst. Man Cybern. A (Syst. Hum.) , 597 - 607

Related content

This is a required field
Please enter a valid email address