© The Institution of Engineering and Technology
This study introduces two novel approaches, pre-partitioning and weight-driven exploration, to enable an efficient learning process in the context of cognitive radio. Learning efficiency is crucial when applying reinforcement learning to cognitive radio since cognitive radio users will cause a higher level of disturbance in the exploration phase. Careful control of the tradeoff between exploration and exploitation for a learning-enabled cognitive radio in order to efficiently learn from the interactions with a dynamic radio environment is investigated. In the pre-partitioning scheme, the potential action space of cognitive radios is reduced by initially randomly partitioning the spectrum in each cognitive radio. Cognitive radios are therefore able to finish their exploration stage faster than more basic reinforcement learning-based schemes. In the weight-driven exploration scheme, exploitation is merged into exploration by taking into account the knowledge gained in exploration to influence action selection, thereby achieving a more efficient exploration phase. The learning efficiency in a cognitive radio scenario is defined and the learning efficiency of the proposed schemes is investigated. The simulation results show that the exploration of cognitive radio is more efficient by using pre-partitioning and weight-driven exploration and the system performance is improved accordingly.
References
-
-
1)
-
Jiang, T., Grace, D., Liu, Y.: `Cognitive radio spectrum sharing schemes with reduced spectrum sensing requirements', The IET Seminar on Cognitive Radio and Software Defined Radios: Technologies and Techniques London, September 2008.
-
2)
-
J. Nie ,
S. Haykin
.
A dynamic channel assignment policy through Q-learning.
IEEE Trans. Neural Netw.
,
6 ,
1443 -
1455
-
3)
-
FCC: ‘Notice of proposed rule making and order’. ET Docket No 03–222, December 2003.
-
4)
-
J. Mitola ,
G.Q. Maguire
.
Cognitive radio: making software radios more personal.
IEEE Pers. Commun.
,
9 ,
13 -
18
-
5)
-
R.S. Sutton ,
A.G. Barto
.
(1998)
Reinforcement learning: an introduction.
-
6)
-
C. Cordeiro ,
K. Challapali ,
D. Birru ,
N.S. Shankar
.
(2005)
IEEE 802.22: the first worldwide wireless standard based on cognitive radios.
-
7)
-
I.F. Akyildiz ,
W. Lee ,
M.C. Vuran ,
S. Mohanty
.
Next generation/dynamic spectrum access/cognitive radio wireless networks: a survey.
Elsevier Comput. Netw.
,
2127 -
2159
-
8)
-
D. Grace ,
T.C. Tozer ,
A.G. Burr
.
Reducing call dropping in distributed dynamic channel assignment algorithms by incorporating power control in wireless ad hoc networks.
IEEE J. Sel. Areas Commun. (Wireless Communication Series)
,
22 ,
2417 -
2428
-
9)
-
Jiang, T., Grace, D., Mitchell, P.D.: `Improvement of Pre-partitioning on reinforcement learning based spectrum sharing', IET Int. Communication Conf. on Wireless Mobile and Computing (CCWMC), 2009.
-
10)
-
Z. Ji ,
K.J.R. Liu
.
Dynamic spectrum sharing: a game theoretical overview.
IEEE Commun. Mag.
,
5 ,
88 -
94
-
11)
-
C. Clancy ,
J. Hecker ,
E. Stuntebeck ,
T. O'Shea
.
Applications of machine learning to cognitive radio networks.
IEEE Wirel. Commun.
,
4 ,
1536 -
1284
-
12)
-
B. Fette
.
(2006)
Cognitive radio technology.
-
13)
-
Thrun, S.B.: `Efficient exploration in reinforcement learning', CS-92–102, Technical, 1992, School of Computer Science, Carnegie-Mellon University.
-
14)
-
L. Kleinrock
.
(1975)
Queueing systems volume i: theory.
-
15)
-
Neel, J.O., Reed, J.: `Game models for cognitive radio algorithm analysis', Software Define Radio Forum Technical Conf., 2004.
-
16)
-
C.K. Tan ,
M.L. Sim ,
T.C. Chuah
.
Game theoretic approach for channel assignment and power control with no-internal-regret learning in wireless ad hoc networks.
IEE Commun.
,
9 ,
1159 -
1169
-
17)
-
K. Tsagkaris ,
A. Katidiotis ,
P. Demestichas
.
Neural network-based learning schemes for cognitive radio systems.
Comput. Commun.
,
14 ,
3394 -
3404
-
18)
-
S. Haykin
.
Cognitive radio: brain-empowered wireless communications.
IEEE J. Sel. Areas Commun.
,
2 ,
201 -
220
-
19)
-
J. Mitola
.
(2000)
Cognitive radio: an integrated agent architecture for software defined radio.
-
20)
-
‘Cognitive radio technology: a study for Ofcom’. Summary Report, QinetiQ Ltd, Febreary 2007.
-
21)
-
T. Jiang ,
D. Grace ,
Y. Liu
.
Two stage reinforcement learning based cognitive radio with exploration control.
IET Commun.
,
5 ,
644 -
651
-
22)
-
S.R. Saunders
.
(1999)
Antennas and propagation for wireless communication systems.
-
23)
-
Čabrić, D., Mishra, S.M., Willkomm, D., Brodersen, R., Wolisz, A.: `A cognitive radio approach for usage of virtual unlicensed spectrum', Fourteenth IST Mobile Wireless Communications Summit, June 2005, Dresden, Germany.
-
24)
-
L.P. Kaelbling ,
M.L. Littman ,
A.W. Moore
.
Reinforcement learning: a survey.
J. Artif. Intell. Res.
,
237 -
285
-
25)
-
Jiang, T., Grace, D., Liu, Y.: `Performance of cognitive radio reinforcement spectrum sharing using different weighting factors', Int. Workshop on Cognitive Networks and Communications (COGCOM) in conjunction with CHINACOM Hangzhou, August 2008, China.
-
26)
-
Bublin, M., Pan, J., Kambourov, I., Slanina, P.: `Distributed spectrum sharing by reinforcement and game theory', Fifth Karlsruhe Workshop on Software Radio, March 2008, Karlsruhe, Germany.
-
27)
-
L. Dasilva ,
A. Mackenzie
.
(2007)
Cognitive networks: tutorial.
-
28)
-
J.O. Neel
.
(2007)
Game theory in the analysis and design of cognitive radio networks.
-
29)
-
N. Nie ,
C. Comaniciu
.
Adaptive channel allocation spectrum etiquette for cognitive radio networks.
Mobile Netw. Appl.
,
779 -
797
-
30)
-
Chen, T., Zhang, H., Katz, M., Zhou, Z.: `Swarm intelligence based dynamic control channel assignment in CogMesh', IEEE ICC (IEEE CoCoNet Workshop), May 2008, Beijing, China.
http://iet.metastore.ingenta.com/content/journals/10.1049/iet-com.2010.0258
Related content
content/journals/10.1049/iet-com.2010.0258
pub_keyword,iet_inspecKeyword,pub_concept
6
6