© The Institution of Engineering and Technology
A challenging application of artificial intelligence systems involves the scheduling of traffic signals in multi-intersection vehicular networks. This paper introduces a novel use of a multi-agent system and reinforcement learning (RL) framework to obtain an efficient traffic signal control policy. The latter is aimed at minimising the average delay, congestion and likelihood of intersection cross-blocking. A five-intersection traffic network has been studied in which each intersection is governed by an autonomous intelligent agent. Two types of agents, a central agent and an outbound agent, were employed. The outbound agents schedule traffic signals by following the longest-queue-first (LQF) algorithm, which has been proved to guarantee stability and fairness, and collaborate with the central agent by providing it local traffic statistics. The central agent learns a value function driven by its local and neighbours' traffic conditions. The novel methodology proposed here utilises the Q-Learning algorithm with a feedforward neural network for value function approximation. Experimental results clearly demonstrate the advantages of multi-agent RL-based control over LQF governed isolated single-intersection control, thus paving the way for efficient distributed traffic signal control in complex settings.
References
-
-
1)
-
B. Yegnanarayana
.
(2004)
Artificial neural networks.
-
2)
-
Salkham, A., Cunningham, R., Garg, A., Cahill, V.: `A collaborative reinforcement learning approach to urban traffic control optimization', Proc. 2008 IEEE/WIC/ACM Int. Conf. on Web Intelligence and Intelligent Agent Technology, December 2008, Sydney, Australia, p. 560–566.
-
3)
-
C. Cai ,
C.K. Wong ,
B.G. Heydecker
.
Adaptive traffic signal control using approximate dynamic programming.
Transp. Res. Part C
,
5 ,
456 -
474
-
4)
-
C.J.C.H. Walkins ,
P. Dayan
.
Q-learning.
Mach. Learn.
,
270 -
300
-
5)
-
B. Abdulhai
.
Reinforcement learning for the true adaptive traffic signal control.
J. Transp. Eng.
,
3 ,
278 -
285
-
6)
-
B. Wolshon ,
W. Taylor
.
Analysis of intersection delay under real-time adaptive signal control.
Transp. Res. Part C
,
53 -
72
-
7)
-
X.-H. Yu ,
W.W. Recker
.
Stochastic adaptive control model for traffic signal systems.
Transp. Res.
,
4 ,
263 -
282
-
8)
-
C. Gershenson
.
Self-organizing traffic lights.
Complex Syst.
,
1 ,
29 -
53
-
9)
-
M. Papageorgiou ,
M. Ben-Akiva ,
J. Bottom ,
P. Bovy ,
S. Hoogendoorn ,
N. Hounsell ,
A. Kotsialos ,
M. McDonald ,
C. Barnhart ,
G. Laporte
.
ITS and traffic management.
-
10)
-
D.S. Broomhead ,
D. Lowe
.
Multivariable functional interpolation and adaptivenetworks.
Complex Syst.
,
321 -
355
-
11)
-
S. Haykin
.
(1998)
Neural networks: a comprehensive foundation.
-
12)
-
Wiering, M.: `Multi-agent reinforcement learning for traffic light control', Proc. 17th Int. Conf. on Machine Learning, 2000.
-
13)
-
Jayakrishnan, R., Mattingly, S., McNally, M.: `Performance study of SCOOT traffic control system with non-ideal detectorization: field operational test in the city of Anaheim', 80thAnn. Meeting of the Transportation Research Board, 2001, Washington, DC.
-
14)
-
Watkins, C.J.C.H.: `Learning from delayed rewards', 1989, PhD, Cambridge University, Cambridge, UK.
-
15)
-
C. Jacob ,
B. Abdulhai
.
Automated adaptive traffic corridor control using reinforcement learning.
Transp. Res. Rec.: J. Transp. Res. Board
,
1 -
8
-
16)
-
M. Papageorgiou ,
C. Diakaki ,
V. Dinopoulou ,
A. Kotsialos ,
Y. Wang
.
Review of road traffic control strategies.
Proc. IEEE
,
12 ,
2043 -
2067
-
17)
-
D. Srinivasan ,
M.C. Choy
.
Cooperative multi-agent system for coordinated traffic signal control.
Intell. Transp. Syst., IEE Proc.
,
1 ,
41 -
50
-
18)
-
L.-B. de Oliveira ,
E. Camponogara
.
Multi-agent model predictive control of signaling split in urban traffic networks.
Transp. Res.
,
1 ,
120 -
139
-
19)
-
R.S. Sutton ,
A.G. Barto
.
(1998)
Reinforcement learning: an introduction.
-
20)
-
R. Wunderlich ,
C. Liu ,
I. Elhanany ,
T. Urbanik
.
A novel signal scheduling algorithm with quality of service provisioning for an isolate intersection.
IEEE Trans. Intell. Transp. Syst.
,
3 ,
536 -
547
-
21)
-
C. Diakaki ,
M. Papageorgiou ,
K. Aboudolas
.
A multivariable regulator approach to traffic-responsive network-wide signal control.
Control Eng. Pract.
,
183 -
195
-
22)
-
J.S. Albus
.
A theory of cerebellar function.
Math. Biosci.
,
25 -
61
-
23)
-
Fehon, P.K.: `Adaptive traffic signals are we missing the boat?', ITE District 6 Ann. Meeting, 2004, DKS Associates.
-
24)
-
L.P. Kaelbling ,
M.L. Littman
.
Reinforcement learning: a survey.
J. Artif. Intell. Res.
,
279 -
284
http://iet.metastore.ingenta.com/content/journals/10.1049/iet-its.2009.0070
Related content
content/journals/10.1049/iet-its.2009.0070
pub_keyword,iet_inspecKeyword,pub_concept
6
6