Your browser does not support JavaScript!
http://iet.metastore.ingenta.com
1887

Reinforcement learning-based multi-agent system for network traffic signal control

Reinforcement learning-based multi-agent system for network traffic signal control

For access to this article, please select a purchase option:

Buy article PDF
£12.50
(plus tax if applicable)
Buy Knowledge Pack
10 articles for £75.00
(plus taxes if applicable)

IET members benefit from discounts to all IET publications and free access to E&T Magazine. If you are an IET member, log in to your account and the discounts will automatically be applied.

Learn more about IET membership 

Recommend Title Publication to library

You must fill out fields marked with: *

Librarian details
Name:*
Email:*
Your details
Name:*
Email:*
Department:*
Why are you recommending this title?
Select reason:
 
 
 
 
 
IET Intelligent Transport Systems — Recommend this title to your library

Thank you

Your recommendation has been sent to your librarian.

A challenging application of artificial intelligence systems involves the scheduling of traffic signals in multi-intersection vehicular networks. This paper introduces a novel use of a multi-agent system and reinforcement learning (RL) framework to obtain an efficient traffic signal control policy. The latter is aimed at minimising the average delay, congestion and likelihood of intersection cross-blocking. A five-intersection traffic network has been studied in which each intersection is governed by an autonomous intelligent agent. Two types of agents, a central agent and an outbound agent, were employed. The outbound agents schedule traffic signals by following the longest-queue-first (LQF) algorithm, which has been proved to guarantee stability and fairness, and collaborate with the central agent by providing it local traffic statistics. The central agent learns a value function driven by its local and neighbours' traffic conditions. The novel methodology proposed here utilises the Q-Learning algorithm with a feedforward neural network for value function approximation. Experimental results clearly demonstrate the advantages of multi-agent RL-based control over LQF governed isolated single-intersection control, thus paving the way for efficient distributed traffic signal control in complex settings.

References

    1. 1)
      • B. Yegnanarayana . (2004) Artificial neural networks.
    2. 2)
      • Salkham, A., Cunningham, R., Garg, A., Cahill, V.: `A collaborative reinforcement learning approach to urban traffic control optimization', Proc. 2008 IEEE/WIC/ACM Int. Conf. on Web Intelligence and Intelligent Agent Technology, December 2008, Sydney, Australia, p. 560–566.
    3. 3)
      • C. Cai , C.K. Wong , B.G. Heydecker . Adaptive traffic signal control using approximate dynamic programming. Transp. Res. Part C , 5 , 456 - 474
    4. 4)
      • C.J.C.H. Walkins , P. Dayan . Q-learning. Mach. Learn. , 270 - 300
    5. 5)
      • B. Abdulhai . Reinforcement learning for the true adaptive traffic signal control. J. Transp. Eng. , 3 , 278 - 285
    6. 6)
    7. 7)
      • X.-H. Yu , W.W. Recker . Stochastic adaptive control model for traffic signal systems. Transp. Res. , 4 , 263 - 282
    8. 8)
      • C. Gershenson . Self-organizing traffic lights. Complex Syst. , 1 , 29 - 53
    9. 9)
      • M. Papageorgiou , M. Ben-Akiva , J. Bottom , P. Bovy , S. Hoogendoorn , N. Hounsell , A. Kotsialos , M. McDonald , C. Barnhart , G. Laporte . ITS and traffic management.
    10. 10)
      • D.S. Broomhead , D. Lowe . Multivariable functional interpolation and adaptivenetworks. Complex Syst. , 321 - 355
    11. 11)
      • S. Haykin . (1998) Neural networks: a comprehensive foundation.
    12. 12)
      • Wiering, M.: `Multi-agent reinforcement learning for traffic light control', Proc. 17th Int. Conf. on Machine Learning, 2000.
    13. 13)
      • Jayakrishnan, R., Mattingly, S., McNally, M.: `Performance study of SCOOT traffic control system with non-ideal detectorization: field operational test in the city of Anaheim', 80thAnn. Meeting of the Transportation Research Board, 2001, Washington, DC.
    14. 14)
      • Watkins, C.J.C.H.: `Learning from delayed rewards', 1989, PhD, Cambridge University, Cambridge, UK.
    15. 15)
      • C. Jacob , B. Abdulhai . Automated adaptive traffic corridor control using reinforcement learning. Transp. Res. Rec.: J. Transp. Res. Board , 1 - 8
    16. 16)
    17. 17)
      • D. Srinivasan , M.C. Choy . Cooperative multi-agent system for coordinated traffic signal control. Intell. Transp. Syst., IEE Proc. , 1 , 41 - 50
    18. 18)
      • L.-B. de Oliveira , E. Camponogara . Multi-agent model predictive control of signaling split in urban traffic networks. Transp. Res. , 1 , 120 - 139
    19. 19)
      • R.S. Sutton , A.G. Barto . (1998) Reinforcement learning: an introduction.
    20. 20)
      • R. Wunderlich , C. Liu , I. Elhanany , T. Urbanik . A novel signal scheduling algorithm with quality of service provisioning for an isolate intersection. IEEE Trans. Intell. Transp. Syst. , 3 , 536 - 547
    21. 21)
      • C. Diakaki , M. Papageorgiou , K. Aboudolas . A multivariable regulator approach to traffic-responsive network-wide signal control. Control Eng. Pract. , 183 - 195
    22. 22)
      • J.S. Albus . A theory of cerebellar function. Math. Biosci. , 25 - 61
    23. 23)
      • Fehon, P.K.: `Adaptive traffic signals are we missing the boat?', ITE District 6 Ann. Meeting, 2004, DKS Associates.
    24. 24)
      • L.P. Kaelbling , M.L. Littman . Reinforcement learning: a survey. J. Artif. Intell. Res. , 279 - 284
http://iet.metastore.ingenta.com/content/journals/10.1049/iet-its.2009.0070
Loading

Related content

content/journals/10.1049/iet-its.2009.0070
pub_keyword,iet_inspecKeyword,pub_concept
6
6
Loading
This is a required field
Please enter a valid email address