Your browser does not support JavaScript!

Two-stage reinforcement-learning-based cognitive radio with exploration control

Two-stage reinforcement-learning-based cognitive radio with exploration control

For access to this article, please select a purchase option:

Buy article PDF
(plus tax if applicable)
Buy Knowledge Pack
10 articles for $120.00
(plus taxes if applicable)

IET members benefit from discounts to all IET publications and free access to E&T Magazine. If you are an IET member, log in to your account and the discounts will automatically be applied.

Learn more about IET membership 

Recommend Title Publication to library

You must fill out fields marked with: *

Librarian details
Your details
Why are you recommending this title?
Select reason:
IET Communications — Recommend this title to your library

Thank you

Your recommendation has been sent to your librarian.

This study presents a novel two-stage reinforcement-learning-based algorithm for distributed cognitive radio (CR) spectrum sharing. The traditional reinforcement-learning model is modified in order to be applied in a fully distributed CR scenario. CRs are able to discover the best available resources autonomously by utilising learning, which results in significantly improved performance, while reducing the need for spectrum sensing. Instead of sensing all available spectrum arbitrarily, the scheme is designed to share the spectrum based on an optimal spectrum sharing strategy, which is discovered by the CR agents from their trial-and-error interactions with the wireless communication environment. On the other hand, the inherent exploration against exploitation trade-off seen in reinforcement learning is also examined in the context of CR. A ‘warm-up’ stage is proposed to effectively control the exploration phase of the learning process. A better system performance can be expected by carefully balancing the tradeoff between exploration and exploitation. The benefit of applying a warm-up stage is demonstrated. Comparisons of system performance using different warm-up strategies are also given to illustrate their impact on the spectrum sharing process.


    1. 1)
      • Senouci, S.-M., Pujolle, G.: `Dynamic channel assignment in cellular networks: a reinforcement learning solution', Int. Conf. Telecommunications, February 2003.
    2. 2)
      • Jiang, T., Grace, D., Liu, Y.: `Performance of cognitive radio reinforcement spectrum sharing using different weighting factors', Int. Workshop on Cognitive Networks and Communications (COGCOM2008) in conjunction with CHINACOM’08, August 2008, Hangzhou, China.
    3. 3)
    4. 4)
    5. 5)
    6. 6)
      • J. Nie , S. Haykin . A dynamic channel assignment policy through Q-learning. IEEE Trans. Neural Netw. , 6 , 1443 - 1455
    7. 7)
      • Neel, J.O., Buehrer, R.M., Reed, B.H., Gilles, R.P.: `Game theoretic analysis of a network of cognitive radios', 45thMidwest Symp. on Circuits and Systems, August 2002.
    8. 8)
      • L.P. Kaelbling , M.L. Littman , A.W. Moore . Reinforcement learning: a survey. J. Artif. Intell. Res. , 237 - 285
    9. 9)
    10. 10)
      • S.R. Saunders . (1999) Antennas and propagation for wireless communication systems.
    11. 11)
    12. 12)
      • T. Mitchell . (1997) Machine learning.
    13. 13)
      • Leith, D.J., Clifford, P.: `Convergence of distributed learning algorithms for optimal wireless channel allocation', 45thIEEE Conf. Decision and Control, 2006, San Diego.
    14. 14)
      • R.S. Sutton , A.G. Barto . (1998) Reinforcement learning: an introduction.
    15. 15)
      • Bublin, M., Pan, J., Kambourov, I., Slanina, P.: `Distributed spectrum sharing by reinforcement and game theory', Fifth Karlsruhe Workshop on Software Radio, March 2008, Karlsruhe, Germany.
    16. 16)
      • B. Fette . (2006) Cognitive radio technology.
    17. 17)
    18. 18)
      • Neel, J.O., Reed, J.: `Game models for cognitive radio algorithm analysis', Software Define Radio Forum Technical Conf., 2004.
    19. 19)
      • FCC: ‘Notice of proposed rule making and order’, December 2003, ET Docket No. 03-222.
    20. 20)
      • Jiang, T., Grace, D., Liu, Y.: `Cognitive radio spectrum sharing schemes with reduced spectrum sensing requirements', The IET Seminar on Cognitive Radio and Software Defined Radios: Technologies and Techniques, September 2008, London.
    21. 21)
      • Cordeiro, C., Challapali, K., Birru, D., Sai Shankar, N.: `IEEE 802.22: the first worldwide wireless standard based on cognitive radios', Dynamic Spectrum Access Networks, 2005, DySPAN 2005.
    22. 22)
      • G. Chouinard . (2009) FCC R&O 08-260 proposed text for antenna height.

Related content

This is a required field
Please enter a valid email address