access icon free QoE-aware Q-learning resource allocation for NOMA wireless multimedia communications

© The Institution of Engineering and Technology
Received 13/02/2020, Accepted 17/06/2020, Revised 31/05/2020, Published 14/07/2020

Hybrid control traffic and multimedia flow over emerging non-orthogonal multiple access (NOMA) could provide both very low latency and very high bandwidth. In this study, a Q-learning-based resource allocation scheme is proposed to improve the quality of experience (QoE) for NOMA user equipment (UE) in downlink wireless multimedia communications. In the proposed framework, the utility is modelled as the QoE with regard to communication resource cost, where UE acts as the agent in the reinforcement Q-learning. UE observes the wireless channel states and takes resource allocation actions based on the immediate reward of QoE gain and communication cost. In addition, benefiting from the NOMA communications, the authors propose to solve the multiple agent reinforcement learning problems with the simplified sequential single agent reinforcement learning (SARL) approach. The numerical simulation results demonstrate the efficiency of the proposed Q-QoE resource allocation framework and prove that the UE would obtain desirable QoE performance with the SARL scheme.

Inspec keywords: learning (artificial intelligence); telecommunication congestion control; multi-access systems; multi-agent systems; resource allocation; telecommunication computing; synchronisation; mobile radio; quality of experience; multimedia communication

Other keywords: QoE gain; quality of experience; QoE-aware Q-learning resource allocation; sequential single agent reinforcement learning approach; multimedia flow; NOMA wireless multimedia communications; hybrid control traffic; wireless channel states; SARL scheme; Q-QoE resource allocation framework; nonorthogonal multiple access; multiple agent reinforcement learning problems; communication resource cost; NOMA user equipment; reinforcement Q-learning

Subjects: Control applications in radio and radar; Knowledge engineering techniques; Multiple access communication; Communications computing; Multimedia communications; Mobile radio systems

References

    1. 1)
      • 3. Xu, Y., Hu, R.Q., Wei, L., et al: ‘QoE-aware mobile association and resource allocation over wireless heterogeneous networks’. Proc. IEEE Global Communications Conf. (GLOBECOM), Austin, USA, December 2014.
    2. 2)
      • 8. He, S., Wang, W.: ‘Least repetition first transmission (LRFT): QoE-oriented repetition relay matrix analysis for wireless multimedia multicast services’. Proc. IEEE Int. Conf. on Communications Workshops (ICC Workshops), Kansas City, USA, 2018.
    3. 3)
      • 24. Buşoniu, L., Babuška, R., De Schutter, B.: ‘Multi-agent reinforcement learning: an overview’ in Dipti, S. (Ed.): ‘Innovations in multi-agent systems and applications-1’ (Springer, New York, USA, 2010), pp. 183–221.
    4. 4)
      • 23. He, S., Wang, W.: ‘Context-aware QoE-price equilibrium for wireless multimedia relay communications using Stackelberg game’. Proc. IEEE Int. Conf. on Computer Communications (INFOCOM), SCAN Workshop, Atlanta, USA, May 2017.
    5. 5)
      • 16. Li, G., Feng, B., Zhou, H., et al: ‘Adaptive service function chaining mappings in 5G using deep Q-learning’, Comput. Commun., 2020, 152, pp. 305–315.
    6. 6)
      • 14. Jiang, C., Zhang, H., Ren, Y., et al: ‘Machine learning paradigms for next-generation wireless networks’, IEEE Wirel. Commun., 2017, 24, (2), pp. 98–105.
    7. 7)
      • 1. CISCO: ‘Cisco visual networking index: global mobile data traffic forecast update, 2016–2021 white paper’. Cisco Visual Networking Index, San Jose, CA, USA, Cisco White Paper, 28 March 2017.
    8. 8)
      • 9. Benjebbour, A., Saito, K., Li, A., et al: ‘Non-orthogonal multiple access (NOMA): concept, performance evaluation and experimental trials’. Proc. IEEE Int. Conf. on Wireless Networks and Mobile Communications (WINCOM), Marrakech, Morocco, October 2015.
    9. 9)
      • 11. He, S., Wang, W.: ‘A QoE-optimized power allocation scheme for non-orthogonal multiple access wireless video services’. Proc. IEEE Global Communications Conf. (GLOBECOM), Abu Dhabi, UAE, December 2018.
    10. 10)
      • 20. Ding, Z., Lai, X., Karagiannidis, G.K., et al: ‘A survey on non-orthogonal multiple access for 5G networks: research challenges and future trends’, IEEE J. Sel. Areas Commun., 2017, 35, (10), pp. 2181–2195.
    11. 11)
      • 13. Forster, A.: ‘Machine learning techniques applied to wireless ad-hoc networks: guide and survey’. Proc. IEEE 3rd Int. Conf. on Intelligent Sensors, Sensor Networks and Information Processing, Melbourne, Qld., Australia, 2007, pp. 365–370.
    12. 12)
      • 6. Zhang, W., Wen, Y., Chen, Z., et al: ‘QoE-driven cache management for HTTP adaptive bit rate streaming over wireless networks’, IEEE Trans. Multimed., 2013, 15, (6), pp. 1431–1445.
    13. 13)
      • 10. Higuchi, K., Benjebbour, A.: ‘Non-orthogonal multiple access (NOMA) with successive interference cancellation for future radio access’, IEICE Trans. Commun., 2015, 98, (3), pp. 403–414.
    14. 14)
      • 19. Luo, Y., Shi, Z., Zhou, X., et al: ‘Dynamic resource allocations based on Q-learning for D2D communication in cellular networks’. Proc. 2014 11th Int. Computer Conf. on Wavelet Active Media Technology and Information Processing (ICCWAMTIP), Chengdu, China, 2014.
    15. 15)
      • 18. Ye, H., Li, G.Y.: ‘Deep reinforcement learning for resource allocation in V2V communications’, arXiv preprint arXiv:1711.00968, 2017.
    16. 16)
      • 4. Ding, Z., Liu, Y., Choi, J., et al: ‘Application of non-orthogonal multiple access in LTE and 5G networks’, arXiv preprint arXiv:1511.08610, 2015.
    17. 17)
      • 15. Geurts, P., El Khayat, I., Leduc, G.: ‘A machine learning approach to improve congestion control over wireless computer networks’. Proc. IEEE Int. Conf. on Data Mining, Brighton, UK, November 2004.
    18. 18)
      • 22. Su, Z., Xu, Q., Fei, M., et al: ‘Game theoretic resource allocation in Media cloud with Mobile social users’, IEEE Trans. Multimedia, 2016, 18, (8), pp. 1650–1660.
    19. 19)
      • 12. Anxin, L., Yang, L., Xiaohang, C., et al: ‘Non-orthogonal multiple access (NOMA) for future downlink radio access of 5G’, China Commun., 2015, 12, pp. 28–37.
    20. 20)
      • 2. Piamrat, K., Ksentini, A., Viho, C., et al: ‘QoE-aware vertical handover in wireless heterogeneous networks’. Proc. IEEE Int. Conf. on Wireless Communications and Mobile Computing Conf. (IWCMC), Istanbul, Turkey, July 2011.
    21. 21)
      • 21. He, S., Wang, W.: ‘Spectrum-shared NOMA game-theoretical power requisition in context-aware wireless multimedia communications’. Proc. IEEE Int. Conf. on Computer Communications (INFOCOM), Workshop on SCAN, Honolulu, USA, April 2018.
    22. 22)
      • 7. He, S., Wang, W.: ‘Wireless image relay: prioritized QoE scheduling with simplified space-time coding mode selection’. Int. Conf. on Wireless Algorithms, Systems, and Applications, Guilin, China, 2017.
    23. 23)
      • 25. Mnih, V., Kavukcuoglu, K., Silver, D., et al: ‘Human-level control through deep reinforcement learning’, Nature, 2015, 518, (7540), pp. 529–533.
    24. 24)
      • 5. He, S., Wang, W.: ‘User-centric QoE-driven power and rate allocation for multimedia rebroadcasting in 5G wireless systems’. Proc. IEEE 83rd Vehicular Technology Conf. (VTC Spring), Nanjing, China, 2016.
    25. 25)
      • 17. Yau, K.L.A., Komisarczuk, P., Teal, P.D.: ‘Reinforcement learning for context awareness and intelligence in wireless networks: review, new features and open issues’, J. Netw. Comput. Appl., 2012, 35, (1), pp. 253–267.
http://iet.metastore.ingenta.com/content/journals/10.1049/iet-net.2020.0021
Loading

Related content

content/journals/10.1049/iet-net.2020.0021
pub_keyword,iet_inspecKeyword,pub_concept
6
6
Loading