© The Institution of Engineering and Technology
This study presents a novel two-stage reinforcement-learning-based algorithm for distributed cognitive radio (CR) spectrum sharing. The traditional reinforcement-learning model is modified in order to be applied in a fully distributed CR scenario. CRs are able to discover the best available resources autonomously by utilising learning, which results in significantly improved performance, while reducing the need for spectrum sensing. Instead of sensing all available spectrum arbitrarily, the scheme is designed to share the spectrum based on an optimal spectrum sharing strategy, which is discovered by the CR agents from their trial-and-error interactions with the wireless communication environment. On the other hand, the inherent exploration against exploitation trade-off seen in reinforcement learning is also examined in the context of CR. A ‘warm-up’ stage is proposed to effectively control the exploration phase of the learning process. A better system performance can be expected by carefully balancing the tradeoff between exploration and exploitation. The benefit of applying a warm-up stage is demonstrated. Comparisons of system performance using different warm-up strategies are also given to illustrate their impact on the spectrum sharing process.
References
-
-
1)
-
Senouci, S.-M., Pujolle, G.: `Dynamic channel assignment in cellular networks: a reinforcement learning solution', Int. Conf. Telecommunications, February 2003.
-
2)
-
Jiang, T., Grace, D., Liu, Y.: `Performance of cognitive radio reinforcement spectrum sharing using different weighting factors', Int. Workshop on Cognitive Networks and Communications (COGCOM2008) in conjunction with CHINACOM’08, August 2008, Hangzhou, China.
-
3)
-
S. Haykin
.
Cognitive radio: brain-empowered wireless communications.
IEEE J. Sel. Areas Commun.
,
2 ,
201 -
220
-
4)
-
J. Mitola ,
G.Q. Maguire
.
Cognitive radio: making software radios more personal.
IEEE Pers. Commun.
,
9 ,
13 -
18
-
5)
-
K. Tsagkaris ,
A. Katidiotis ,
P. Demestichas
.
Neural network-based learning schemes for cognitive radio systems.
Comput. Commun.
,
14 ,
3394 -
3404
-
6)
-
J. Nie ,
S. Haykin
.
A dynamic channel assignment policy through Q-learning.
IEEE Trans. Neural Netw.
,
6 ,
1443 -
1455
-
7)
-
Neel, J.O., Buehrer, R.M., Reed, B.H., Gilles, R.P.: `Game theoretic analysis of a network of cognitive radios', 45thMidwest Symp. on Circuits and Systems, August 2002.
-
8)
-
L.P. Kaelbling ,
M.L. Littman ,
A.W. Moore
.
Reinforcement learning: a survey.
J. Artif. Intell. Res.
,
237 -
285
-
9)
-
N. Nie ,
C. Comaniciu
.
Adaptive channel allocation spectrum etiquette for cognitive radio networks.
Mobile Netw. Appl.
,
779 -
797
-
10)
-
S.R. Saunders
.
(1999)
Antennas and propagation for wireless communication systems.
-
11)
-
C. Clancy ,
J. Hecker ,
E. Stuntebeck ,
T. O'Shea
.
Applications of machine learning to cognitive radio networks.
IEEE Wirel. Commun.
,
4 ,
1536 -
1284
-
12)
-
T. Mitchell
.
(1997)
Machine learning.
-
13)
-
Leith, D.J., Clifford, P.: `Convergence of distributed learning algorithms for optimal wireless channel allocation', 45thIEEE Conf. Decision and Control, 2006, San Diego.
-
14)
-
R.S. Sutton ,
A.G. Barto
.
(1998)
Reinforcement learning: an introduction.
-
15)
-
Bublin, M., Pan, J., Kambourov, I., Slanina, P.: `Distributed spectrum sharing by reinforcement and game theory', Fifth Karlsruhe Workshop on Software Radio, March 2008, Karlsruhe, Germany.
-
16)
-
B. Fette
.
(2006)
Cognitive radio technology.
-
17)
-
I.F. Akyildiz ,
W. Lee ,
M.C. Vuran ,
S. Mohanty
.
Next generation/dynamic spectrum access/cognitive radio wireless networks: a survey.
Elsevier Comput. Netw.
,
2127 -
2159
-
18)
-
Neel, J.O., Reed, J.: `Game models for cognitive radio algorithm analysis', Software Define Radio Forum Technical Conf., 2004.
-
19)
-
FCC: ‘Notice of proposed rule making and order’, December 2003, ET Docket No. 03-222.
-
20)
-
Jiang, T., Grace, D., Liu, Y.: `Cognitive radio spectrum sharing schemes with reduced spectrum sensing requirements', The IET Seminar on Cognitive Radio and Software Defined Radios: Technologies and Techniques, September 2008, London.
-
21)
-
Cordeiro, C., Challapali, K., Birru, D., Sai Shankar, N.: `IEEE 802.22: the first worldwide wireless standard based on cognitive radios', Dynamic Spectrum Access Networks, 2005, DySPAN 2005.
-
22)
-
G. Chouinard
.
(2009)
FCC R&O 08-260 proposed text for antenna height.
http://iet.metastore.ingenta.com/content/journals/10.1049/iet-com.2009.0803
Related content
content/journals/10.1049/iet-com.2009.0803
pub_keyword,iet_inspecKeyword,pub_concept
6
6