Your browser does not support JavaScript!
http://iet.metastore.ingenta.com
1887

access icon free Enhanced reinforcement learning assisted dynamic power management model for internet-of-things centric wireless sensor network

© The Institution of Engineering and Technology
Received 10/01/2020, Accepted 03/09/2020, Revised 06/08/2020, Published 09/09/2020

The exponential rise in the demands of the wireless communication system has alarmed industries to achieve more efficient and quality-of-service (QoS) centric wireless communication networks. The decentralised and infrastructure-less nature of wireless sensor networks (WSNs) enable it to be one of the most sought and used wireless network globally. Its cost-efficiency and functional robustness towards low-power lossy networks make it suitable for internet-of-things (IoT) applications. In recent years, IoT technologies have been used in diverse applications, including Smart City Planning and Management (SCPM). Although, mobile-WSN has played a decisive role in IoT enabled SCPM, its routing optimality and power transmission have always remained challenging. Noticeably, major existing researches address mainly on routing optimisation and very few efforts are made towards dynamic power management (DPM) under non-linear network conditions. With this motive, in this study, a highly robust and efficient QoS – centric reinforcement learning-based DPM model has been developed for mobile-WSN to be used in SCPM. Unlike classical reinforcement learning methods, the authors’ proposed advanced reinforcement learning-based DPM model exploits both known and unknown network parameters and state-activity values, including bit-error probability, channel state information, holding time, buffer cost etc. to perform dynamic switching decision. The key objective of the proposed model is to ensure optimal QoS oriented DPM and adaptive switching control to yield reliable transmission with the maximum possible resource utilisation. To achieve it, they proposed model has been developed as a controlled-Markov decision problem by applying hidden Markov model it obtains known and unknown parameters, which are subsequently learnt using an enhanced reinforcement learning to yield maximum resource utilisation while maintaining low buffer cost, holding cost and bit-error probability to retain the QoS provision.

Inspec keywords: hidden Markov models; wireless sensor networks; resource allocation; learning (artificial intelligence); Internet of Things; error statistics; telecommunication power management; optimisation; Internet; quality of service

Other keywords: optimal QoS; enhanced reinforcement; dynamic switching decision; Smart City Planning; SCPM; nonlinear network conditions; internet-of-things centric wireless sensor network; centric reinforcement learning-based DPM model; low-power lossy networks; classical reinforcement learning methods; wireless communication system; highly robust QoS; cost-efficiency; assisted dynamic power management model; network parameters; bit-error probability; adaptive switching control; power transmission; hidden Markov model; IoT technologies; functional robustness; routing optimality; internet-of-things applications; low buffer cost; mobile-WSN

Subjects: Knowledge engineering techniques; Wireless sensor networks; Mobile radio systems; Markov processes; Computer communications; Markov processes; Other computer networks

References

    1. 1)
      • 5. Bogatinoska, D.C., Malekian, R., Trengoska, J., et al: ‘Advanced sensing and internet of things in smart cities’. 2016 39th Int. Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO), Opatija, 2016, pp. 632–637.
    2. 2)
      • 48. Hwang, C.-H., Wu, A.: ‘A predictive system shutdown method for energy saving of event-driven computation’, ACM Trans. Des. Autom. Electron. Syst., 2000, 5, pp. 226–241.
    3. 3)
      • 41. Galzarano, S., Savaglio, C., Liotta, A., et al: ‘Gossiping-based AODV for wireless sensor networks’. Proc. of the 2013 IEEE Int. Conf. on Systems, Man, and Cybernetics, Manchester, UK, 13–16 October 2013, pp. 26–31.
    4. 4)
      • 36. Chincoli, M., Liotta, A.: ‘Transmission power control in WSNs: from deterministic to cognitive methods’. Integration, Interconnection, and Interoperability of IoT Systems, Switzerland, 2018, pp. 39–57.
    5. 5)
      • 31. Felemban, E., Member, S., Lee, C., et al: ‘MMSPEED: multipath multi-SPEED protocol for QoS guarantee of reliability and timeliness in wireless sensor networks’, IEEE Trans. Mob. Comput., 2006, 5, (6), pp. 738–754.
    6. 6)
      • 2. Bharadwaj, A.S., Rego, R., Chowdhury, A.: ‘IoT based solid waste management system: a conceptual approach with an architectural solution as a smart city application’. 2016 IEEE Annual India Conf. (INDICON), Bangalore, 2016, pp. 1–6.
    7. 7)
      • 22. Khanke, K., Sarde, M.: ‘An energy efficient and QoS aware routing protocol for wireless sensor network’, Int. J. Adv. Res. Comput. Commun. Eng., 2015, 4, (7), pp. 355–358.
    8. 8)
      • 26. Jaradat, T., Benhaddou, D., Balakrishnan, M., et al: ‘Energy efficient cross-layer routing protocol in wireless sensor networks based on fuzzy logic’. 2013 9th Int. Wireless Communications and Mobile Computing Conference (IWCMC), Sardinia, 2013, pp. 177–182.
    9. 9)
      • 45. Liang, X., Balasingham, I., Leung, V.C.M.: ‘Cooperative communications with relay selection for QoS provisioning in wireless sensor networks’. Proc. of the 2009 IEEE Global Telecommunications Conf., Honolulu, HI, USA, 30 November–4 December 2009.
    10. 10)
      • 43. Azizi, R.: ‘Consumption of energy and routing protocols in wireless sensor network’, Netw, Protoc. Algorithms, 2016, 8, p. 76.
    11. 11)
      • 1. Sakhardande, P., Hanagal, S., Kulkarni, S.: ‘Design of disaster management system using IoT based interconnected network with smart city monitoring’. 2016 Int. Conf. on Internet of Things and Applications (IOTA), Pune, 2016, pp. 185–190.
    12. 12)
      • 49. Qiu, Q., Tan, Y., Wu, Q.: ‘Stochastic modeling and optimization for robust power management in a partially observable system’. Proc. of the Conf. on Design, Automation and Test in Europe (DATE ‘07), Nice, 2007, pp. 1–6.
    13. 13)
      • 27. Chen, M., Leung, V. C., Mao, S., et al: ‘Directional geographical routing for real-time video communications in wireless sensor networks’, Comput. Commun., 2007, 30, (17), pp. 3368–3383.
    14. 14)
      • 30. Ahmad, I., Ranka, S., Kham, S.U.: ‘Using game theory for scheduling tasks on multi-core processor for simultaneous optimization of performance and energy’. Proc. of the IEEE Int. Symp. on Parallel and Distributed Proc., Miami, FL, 2008, pp. 1–6.
    15. 15)
      • 9. Harmon, R.R., Castro-Leon, E.G., Bhide, S.: ‘Smart cities and the internet of things’. 2015 Portland Int. Conf. on Management of Engineering and Technology (PICMET), Portland, OR, 2015, pp. 485–494.
    16. 16)
      • 8. Sumi, L., Ranga, V.: ‘Sensor enabled internet of things for smart cities’. 2016 Fourth Int. Conf. on Parallel, Distributed and Grid Computing (PDGC), Waknaghat, 2016, pp. 295–300.
    17. 17)
      • 24. Chen, Y., Gao, L., Xing, Y., et al: ‘Cross-layer design for energy-efficient reliable routing in wireless sensor networks’. 2015 11th Int. Conf. on Mobile Ad-hoc and Sensor Networks (MSN), Shenzhen, 2015, pp. 31–36.
    18. 18)
      • 33. Rajan, D., Sabharwal, A., Aazhang, B.: ‘Delay-bounded packet scheduling of bursty traffic over wireless channels’, IEEE Trans. Inf. Theory, 2004, 50, (1), pp. 125–144.
    19. 19)
      • 51. Tan, Y., Qiu, Q.: ‘A framework of stochastic power management using hidden Markov model’. Proc. of the Conf. on Design, Automation and Test in Europe (DATE ‘08), New York, NY, USA2008, pp. 92–97.
    20. 20)
      • 56. Ipek, E., Mutlu, O., Martinez, J.F., et al: ‘Self-optimizing memory controllers: a reinforcement learning approach’. Proc. of the 35th Annual Int. Symp. on Computer Architecture, USA, 2008, pp. 39–50.
    21. 21)
      • 52. Wang, Y., Ma, K., Wang, X.: ‘Temperature-constrained power control for chip multiprocessors with online model estimation’. Proc. of the 36th Annual Int. Symp. on Computer Architecture, (ISCA ‘09), New York, NY, USA, 2009, pp. 314–324.
    22. 22)
      • 18. Kianpisheh, S., Charkari, N.M.: ‘A new approach for power management in sensor node based on reinforcement learning’. 2011 Int. Symp. on Computer Networks and Distributed Systems (CNDS), Tehran, 2011, pp. 158–163.
    23. 23)
      • 10. Zanella, A., Bui, N., Castellani, A., et al: ‘Internet of things for smart cities’, IEEE Internet Things J., 2014, 1, (1), pp. 22–32.
    24. 24)
      • 14. Minoli, D., Occhiogrosso, B.: ‘Mobile IPv6 protocols and high efficiency video coding for smart city IoT applications’. 2017 13th Int. Conf. and Expo on Emerging Technologies for a Smarter World (CEWIT), Stony Brook, NY, 2017, pp. 1–6.
    25. 25)
      • 19. Jin, A., Song, W., Ju, P., Zhou, D.: ‘Energy-aware cooperation strategy with uncoordinated group relays for delay-sensitive services’, IEEE Trans. Veh. Technol., 2014, 63, (5), pp. 2104–2114.
    26. 26)
      • 21. Shiny, V. A., Nagarajan, V.: ‘Energy efficient routing protocol for mobile wireless sensor network’, Int. J. Comput. Appl., 2012, 43, (21), pp. 1–5.
    27. 27)
      • 38. Kulkarni, R.V., Forster, A., Venayagamoorthy, G.K.: ‘Computational intelligence in wireless sensor networks: a survey’, IEEE Commun. Surv. Tutor., 2011, 13, pp. 68–96.
    28. 28)
      • 7. Santos, P.M., Rodrigues, J.G., Cruz, S.B., et al: ‘Portolivinglab: an IoT-based sensing platform for smart cities’, IEEE Internet Things J., 2018, 5, (2), (99), pp. 523–532.
    29. 29)
      • 20. Sen, J., Ukil, A.: ‘An adaptable and QoS-aware routing protocol for wireless sensor networks’. 1st Int. Conf. Wireless Communication, Vehicular Technology, Information Theory and Aerospace & Electronics Systems Technology, Aalborg, 17–20 May 2009, pp. 767–771.
    30. 30)
      • 50. Rosing, T.S., Benini, L., Glynn, P., et al: ‘Event-driven power management’, IEEE Trans. Comput. Aided Des. Integr. Circuits Syst., 2001, 20, pp. 840–857.
    31. 31)
      • 25. Shareef, A., Shareef, A., Zhu, Y.: ‘Optrix: energy aware cross layer routing using convex optimization in wireless sensor networks’. 2015 IEEE Int. Conf. on Networking, Architecture and Storage (NAS), Boston, MA, 2015, pp. 141–150.
    32. 32)
      • 46. Liang, X., Chen, M., Leung, V.C.M., et al: ‘Soft QoS provisioning for wireless sensor networks: a cooperative communications approach’. Proc. of the 2010 5th International ICST Conf. on Communications and Networking in China, Beijing, China, 25–27 August 2010. pp. 1–8.
    33. 33)
      • 29. Othman, J.B., Yahya, B.: ‘Energy efficient and QoS based routing protocol for wireless sensor networks’, J. Parallel Distrib. Comput., 2010, 70, (8), pp. 849–857.
    34. 34)
      • 28. Zhuo, S., Wang, Z., Song, Y.-Q., et al: ‘iQueue-MAC: a traffic adaptive duty-cycled’.
    35. 35)
      • 17. Shen, H., Tan, Y., Lu, J., et al: ‘Achieving autonomous power management using reinforcement learning’, ACM Trans. Des. Autom. of Electron. Syst. (TODAES), 2013, 18, (2), pp. 1–32.
    36. 36)
      • 39. Galzarano, S., Liotta, A., Fortino, G.: ‘QL-MAC: A Q-learning based MAC for wireless sensor networks’. Algorithms and Architectures for Parallel Processing; ICA3PP 2013, Switzerland, (LNCS, 8286), pp. 267–275.
    37. 37)
      • 23. Zhuo, S., Wang, Z., Song, Y., et al: ‘iQueue-MAC: a traffic adaptive duty-cycled MAC protocol with dynamic slot allocation’. IEEE Int. Conf. on Sensing, Communications and Networking (SECON), New Orleans, LA, 2013, pp. 95–103.
    38. 38)
      • 6. Murthy, A., Han, D., Jiang, D., et al: ‘Lighting-enabled smart city applications and ecosystems based on the IoT’. 2015 IEEE 2nd World Forum on Internet of Things (WF-IoT), Milan, 2015, pp. 757–763.
    39. 39)
      • 47. Gummeson, J., Ganesan, D., Corner, M.D., et al: ‘An adaptive link layer for heterogeneous multi-radio mobile sensor networks’, IEEE J. Sel. Areas Commun., 2010, 28, pp. 1094–1104.
    40. 40)
      • 54. Cai, L., Pettis, N., Lu, Y.: ‘Joint power management of memory and disk under performance constraints’, IEEE Trans. Comput. Aided Des. Integr. Circuits Syst., 2006, 25, pp. 2697–2711.
    41. 41)
      • 13. Dalla Cia, M., Mason, F., Peron, D., et al: ‘Mobility-aware handover strategies in smart cities’. 2017 Int Symp. on Wireless Communication Systems (ISWCS), Bologna, 2017, pp. 438–443.
    42. 42)
      • 12. Daneels, G., Municio, E., Spaey, K., et al: ‘Real-time data dissemination and analytics platform for challenging IoT environments’. 2017 Global Information Infrastructure and Networking Sym (GIIS), St. Pierre, 2017, pp. 23–30.
    43. 43)
      • 3. Nirde, K., Mulay, P.S., Chaskar, U.M.: ‘IoT based solid waste management system for smart city’. 2017 Int. Conf. on Intelligent Computing and Control Systems, Madurai, 2017, pp. 666–669.
    44. 44)
      • 40. Yau, K.L.A., Goh, H.G., Chieng, D., et al: ‘Application of reinforcement learning to wireless sensor networks: models and algorithms’, Computing, 2015, 97, pp. 1045–1075.
    45. 45)
      • 58. Tesauro, G., Das, R., Jong, N., Bennani, M.: ‘A hybrid reinforcement learning approach to autonomic resource allocation’. Proc. of 3rd IEEE Int. Conf. on Autonomic Computing (ICAC ‘06), 2006, pp. 65–73.
    46. 46)
      • 44. Chincoli, M., Syed, A.A., Exarchakos, G., et al: ‘Power control in wireless sensor networks with variable interference’, Mob. Inf. Syst., 2016, 2016, pp. 1–10.
    47. 47)
      • 35. Bosman, H.H., Iacca, G., Tejada, A., et al: ‘Spatial anomaly detection in sensor networks using neighborhood information’, Inf. Fus., 2017, 33, pp. 41–56.
    48. 48)
      • 11. Datta, S.K., Bonnet, C.: ‘Internet of things and M2M communications as enablers of smart city initiatives’. 2015 9th Int. Conf. on Next Generation Mobile Applications, Services and Technologies, Cambridge, 2015, pp. 393–398.
    49. 49)
      • 37. Tesauro, G., Das, R., Chan, H., et al: ‘Managing power consumption and performance of computing systems using reinforcement learning’. Proc. of the 21st Annual Conf. on Neural Information Processing Systems (NIPS ‘07), Vancouver, B.C., Canada, 2007, pp. 1497–1504.
    50. 50)
      • 15. Wang, M., Wu, J., Li, G., et al: ‘Toward mobility support for information-centric IoV in smart city using fog computing’. 2017 IEEE Int. Conf. on Smart Energy Grid Engineering (SEGE), Oshawa, ON, 2017, pp. 357–361.
    51. 51)
      • 16. Hosahalli, D., Srinivas, K.G.: ‘Cross-layer routing protocol for event-driven M2M communication in IoT-assisted smart city planning and management: CWSN-eSCPM’, IET Wirel. Sens. Syst., 2019, 10, (1), pp. 1–12.
    52. 52)
      • 7. Santos, P.M., Rodrigues, J.G., Cruz, S.B., et al: ‘Portolivinglab: an IoT-based sensing platform for smart cities’, IEEE Internet Things J., 2018, 5, (2), pp. 523–532.
    53. 53)
      • 34. Bosman, H.H.W.J., Iacca, G., Tejada, A., et al: ‘Ensembles of incremental learners to detect anomalies in ad hoc sensor networks’, Ad Hoc Netw., 2015, 35, pp. 14–36.
    54. 54)
      • 4. ReportLinker (April 09, 2020): ‘Internet of things (IoT) market in manufacturing – growth, trends, and forecast (2020–2025)’. Available at: https://www.globenewswire.com/news-release/2020/04/09 /2014196/0/en/Internet-of-Things-IoT-Market-In-Manufacturing-Growth-Trends-and-Forecast-2020-2025.html, Accessed 19 May 2020.
    55. 55)
      • 42. Nitti, M., Murroni, M., Fadda, M., et al: ‘Exploiting social internet of things features in cognitive radio’, IEEE Access, 2016, 4, pp. 9204–9212.
    56. 56)
      • 57. Dhiman, G., Rosing, T.S.: ‘System-level power management using online learning’, IEEE Trans. Comput. Aided Des. Integr. Circuits Syst., 2009, 28, pp. 676–689.
    57. 57)
      • 55. Martinez, J.F., Ipek, E.: ‘Dynamic multicore resource management: a machine learning approach’, IEEE Micro, 2009, 29, pp. 8–17.
    58. 58)
      • 32. Schaar, M.V., Turaga, D.S.: ‘Cross-layer packetization and retransmission strategies for delay-sensitive wireless multimedia transmission’, IEEE Trans. Multimedia, 2007, 9, (1), pp. 185–197.
    59. 59)
      • 53. Zanini, F., Atienza, D., Benini, L., et al: ‘Multicore thermal management with model predictive control’. Proc. of the 19th European Conf. on Circuit Theory and Design, 2009, pp. 90–95.
http://iet.metastore.ingenta.com/content/journals/10.1049/iet-com.2020.0026
Loading

Related content

content/journals/10.1049/iet-com.2020.0026
pub_keyword,iet_inspecKeyword,pub_concept
6
6
Loading
Print this page
This is a required field
Please enter a valid email address