This study proposes an approximate dynamic programming (ADP) method for a stochastic home energy management system (HEMS) that aims to minimise the electricity cost and discomfort of a household under uncertainties. The study focuses on a HEMS that optimally schedules heating, ventilation, and air conditioning, water heater, and electric vehicle, while accounting for uncertainties in outside temperature, hot water usage, and non-controllable net load. The authors approach the ADP-based HEMS via an effective combination of Sobol sampling backward induction and a K–D tree nearest neighbour techniques for the value function approximation. A subset of possible states is sampled and used to create an approximation of the value of being in aggregated states. They compare the ADP approach with other prevailing HEMS methods, including dynamic programming (DP) and mixed-integer linear programming (MILP), in a model predictive control framework. Simulation results show that the proposed ADP approach can yield near-optimal appliance schedules under uncertainties when finely discretised. Merits and drawbacks of the proposed ADP method in comparison with DP and MILP are also revealed.

References

1. 1)
  - 41. Hoke, A., Brissette, A., Simith, K., et al: ‘Accounting for lithium-ion battery degradation in electric vehicle charging optimization’, IEEE J. Emerging Sel. Top. Power Electron., 2014, 2, (3), pp. 691–700.
2. 2)
  - 21. Livengood, D., Larson, R.: ‘The energy box: locally automated optimal control of residential electricity usage’, Serv. Sci., 2009, 1, (1), pp. 1–16.
3. 3)
  - 44. Bukenberger, J., Palmintier, B.: ‘Stochastic generation capacity expansion planning with approximate dynamic programming’. 2018 IEEE/PES Transmission and Distribution Conf. and Exposition (T&D), Denver, USA, April 2018, pp. 1–5.
4. 4)
  - 28. Huang, T., Liu, D.: ‘Residential energy system control and management using adaptive dynamic programming’. ‘The 2011 Int. Joint Conf. on Neural Networks’, San Jose, USA, October 2011, pp. 119–124.
5. 5)
  - 30. Liu, D., Xu, Y., Wei, Q., et al: ‘Residential energy scheduling for variable weather solar energy based on adaptive dynamic programming’. IEEE/CAA J. Autom. Sinica, 2018, 5, (1), pp. 36–46.
6. 6)
  - 46. Berg, M., Cheong, O., Kreveld, M., et al: ‘Computational geometry: algorithm and applications’ (Springer, New York, USA, 2008, 3rd edn.).
7. 7)
  - 35. Keerthisinghe, C., Verbič, G., Chapman, A.C.: ‘A fast technique for smart home management: ADP with temporal difference learning’, IEEE Trans. Smart Grid, 2016, 9, (4), pp. 3291–3303.
8. 8)
  - 33. Borghesan, F., Vignali, R., Piroddi, L., et al: ‘Approximate dynamic programming-based control of a building cooling system with thermal storage’. IEEE PES ISGT Europe 2013, Lyngby, Denmark, October 2013, pp. 1–5.
9. 9)
  - 51. Mathieu, J.L.: ‘Modeling, analysis, and control of demand response resources’, PhD thesis, UC Berkeley, 2012.
10. 10)
  - 2. Pratt, A., Krishnamurthy, D., Ruth, M., et al: ‘Transactive home energy management systems: the impact of their proliferation on the electric grid’, IEEE Electrification Mag., 2016, 4, (4), pp. 8–14.
11. 11)
  - 10. Chang, T.H., Alizadeh, M., Scaglione, A.: ‘Real-time power balancing via decentralized coordinated home energy scheduling’, IEEE Trans. Smart Grid, 2013, 4, (3), pp. 1490–1504.
12. 12)
  - 6. Shafie-Khah, M., Siano, P.: ‘A stochastic home energy management system considering satisfaction cost and response fatigue’, IEEE Trans. Ind. Inf., 2018, 14, (2), pp. 629–638.
13. 13)
  - 34. Ceriani, N., Vignali, R., Piroddi, L., et al: ‘An approximate dynamic programming approach to the energy management of a building cooling system’. 2013 European Control Conf. (ECC), Zurich Switzerland, July 2013, pp. 2026–2031.
14. 14)
  - 49. Wu, H., Palmintier, B., Krishnamurthy, D.: ‘DOE CODE: project metadata for code ID 12483’. Available at https://www.osti.gov/doecode/biblio/12483, accessed May 2020.
15. 15)
  - 25. Bertsekas, D.P.: ‘Dynamic programming and optimal control: approximate dynamic programming’ (Athena Scientific, Nashua, NH, USA, 2012, 4th edn.).
16. 16)
  - 38. Wu, H., Pratt, A., Chakraborty, S.: ‘Stochastic optimal scheduling of residential appliances with renewable energy sources’. 2015 IEEE Power Energy Society General Meeting, Denver, USA, July 2015, pp. 1–5.
17. 17)
  - 32. Zhu, J., Mo, X., Zhu, T., et al: ‘Real-time stochastic operation strategy of a microgrid using approximate dynamic programming-based spatiotemporal decomposition approach’, IET Renew. Power Gener., 2019, 9, (16), pp. 3061–3070.
18. 18)
  - 12. Ahmed, M.S., Mohamed, A., Khatib, T., et al: ‘Real time optimal schedule controller for home energy management system using new binary backtracking search algorithm’, Energy Build., 2017, 138, pp. 215–227.
19. 19)
  - 14. Carli, R., Dotoli, M.: ‘Decentralized control for residential energy management of a smart users’ microgrid with renewable energy exchange’, IEEE/CAA J. Autom. Sinica, 2019, 6, (3), pp. 641–656.
20. 20)
  - 24. Powell, W.B.: ‘Approximate dynamic programming: solving the curses of dimensionality (Wiley series in probability and statistics)’ (Wiley, Hoboken, NJ, USA, 2011, 2nd edn.).
21. 21)
  - 5. Jo, H.C., Kim, S., Joo, S.K.: ‘Smart heating and air conditioning scheduling method incorporating customer convenience for home energy management system’, IEEE Trans. Consum. Electron., 2013, 59, (2), pp. 316–322.
22. 22)
  - 8. Pedrasa, M.A.A., Spooner, T.D., MacGill, I.F.: ‘Coordinated scheduling of residential distributed energy resources to optimize smart home energy services’, IEEE Trans. Smart Grid, 2010, 1, (2), pp. 134–143.
23. 23)
  - 43. Senthilnath, J., Raj, A., Omkar, S.N., et al: ‘Quasi-based hierarchical clustering for land cover mapping using satellite images’, Adv. Intell. Syst. Comput., 2013, 202, (12), pp. 53–64.
24. 24)
  - 26. Keerthisinghe, C., Verbic, G., Chapman, A. C.: ‘Energy management of PV–storage systems: ADP approach with temporal difference learning’. 2016 Power Systems Computation Conf. (PSCC), Genova, Italy, June 2016, pp. 1–7.
25. 25)
  - 27. Wei, Q., Liu, D., Shi, G., et al: ‘Multibattery optimal coordination control for home energy management systems via distributed iterative adaptive dynamic programming’, IEEE Trans. Ind. Electron., 2015, 62, (7), pp. 4203–4214.
26. 26)
  - 40. Divya, K.C., Ostergaard, J.: ‘Battery energy storage technology for power systems – an overview’, Electr. Power Syst. Res., 2009, 79, (4), pp. 511–520.
27. 27)
  - 3. Faqiry, M.N., Wang, L., Wu, H.: ‘HEMS-enabled transactive flexibility in real-time operation of three-phase unbalanced distribution systems’, J. Mod. Power Syst. Clean Energy, 2019, 7, (6), pp. 1434–1449.
28. 28)
  - 42. Sobol, I. M., Levitan, Y. L.: ‘A pseudo-random number generator for personal computers’, Comput. Math. Appl., 1999, 37, (4), pp. 33–40.
29. 29)
  - 18. Hubert, T., Grijalva, S.: ‘Modeling for residential electricity optimization in dynamic pricing environments’, IEEE Trans. Smart Grid, 2012, 3, (4), pp. 2224–2231.
30. 30)
  - 13. Chen, S., Liu, T., Gao, F., et al: ‘Butler, not servant: a human-centric smart home energy management system’, IEEE Commun. Mag., 2017, 55, (2), pp. 27–33.
31. 31)
  - 1. Pratt, A., Banerjee, B., Nemarundwe, T.: ‘Proof-of-concept home energy management system autonomously controlling space heating’. 2013 IEEE Power Energy Society General Meeting, Vancouver, Canada, July 2013, pp. 1–5.
32. 32)
  - 29. Squartini, S., Fuselli, D., Boaro, M., et al: ‘Home energy resource scheduling algorithms and their dependency on the battery model’. 2013 IEEE Computational Intelligence Applications in Smart Grid (CIASG), Singapore, September 2013, pp. 122–129.
33. 33)
  - 9. Deng, R., Yang, Z., Chen, J., et al: ‘Load scheduling with price uncertainty and temporally-coupled constraints in smart grids’, IEEE Trans. Power Syst., 2014, 29, (6), pp. 2823–2834.
34. 34)
  - 11. Shakeri, M., Shayestegan, M., Abunima, H., et al: ‘An intelligent system architecture in home energy management systems (HEMS) for efficient demand response in smart grid’, Energy Build., 2017, 138, pp. 154–164.
35. 35)
  - 7. Zhao, Z., Lee, W.C., Shin, Y., et al: ‘An optimal power scheduling method for demand response in home energy management system’, IEEE Trans. Smart Grid, 2013, 4, (3), pp. 1391–1400.
36. 36)
  - 39. ISO 7730:2005: ‘Ergonomics of the thermal environment — analytical determination and interpretation of thermal comfort using calculation of the PMV and PPD indices and local thermal comfort criteria’, 2005.
37. 37)
  - 19. Chen, X., Wei, T., Hu, S.: ‘Uncertainty-aware household appliance scheduling considering dynamic electricity pricing in smart home’, IEEE Trans. Smart Grid, 2013, 4, (2), pp. 932–941.
38. 38)
  - 45. Christensen, P., Kensler, A., Kilpatrick, C.: ‘Progressive multi-jittered sample sequences’, Comput. Graph. Forum, 2018, 37, (4), pp. 21–33.
39. 39)
  - 36. Wang, L.: ‘An approximate dynamic programming based home energy management system and its impact on the distribution system’, Master thesis, Kansas State University, 2019.
40. 40)
  - 16. Chen, Z., Wu, L., Fu, Y.: ‘Real-time price-based demand response management for residential appliances via stochastic optimization and robust optimization’, IEEE Trans. Smart Grid, 2012, 3, (4), pp. 1822–1831.
41. 41)
  - 48. Dupačová, J., Gröwe-Kuska, N., Römisch, W.: ‘Scenario reduction in stochastic programming’, Math. Program., 2003, 95, (3), pp. 493–511.
42. 42)
  - 15. Bozchalui, M.C., Hashmi, S.A., Hassen, H., et al: ‘Optimal operation of residential energy hubs in smart grids’, IEEE Trans. Smart Grid, 2012, 3, (4), pp. 1755–1766.
43. 43)
  - 20. Yu, Z., Jia, L., Murphy-Hoye, M. C., et al: ‘Modeling and stochastic control for home energy management’, IEEE Trans. Smart Grid, 2013, 4, (4), pp. 2244–2255.
44. 44)
  - 52. Wu, H., Krad, I., Florita, A., et al: ‘Stochastic multi-timescale power system operations with variable wind generation’, IEEE Trans. Power Syst., 2017, 32, (5), pp. 3325–3337.
45. 45)
  - 17. Conejo, A.J., Morales, J.M., Baringo, L.: ‘Real-time demand response model’, IEEE Trans. Smart Grid, 2010, 1, (3), pp. 236–242.
46. 46)
  - 4. Byun, J., Hong, I., Park, S.: ‘Intelligent cloud home energy management system using household appliance priority based scheduling based on prediction of renewable energy capability’, IEEE Trans. Consum. Electron., 2012, 58, (4), pp. 1194–1201.
47. 47)
  - 37. Faqiry, M.N., Wang, L., Wu, H., et al: ‘ADP-based Home energy management system: A case study using DYNAMO’. IEEE Power Energy Society General Meeting (PESGM), Portland, OR, USA, August 2018, pp. 1–5.
48. 48)
  - 47. Friedman, S., Bentley, J., Finkel, R.: ‘An algorithm for finding best matches in logarithmic expected time’, ACM Trans. Math. Softw., 1977, 3, (3), pp. 209–226.
49. 49)
  - 23. Liu, L., Zhou, Y., Liu, Y., et al: ‘Dynamic programming based game theoretic algorithm for economical multi-user smart home scheduling’. 2014 IEEE 57th Int. Midwest Symp. on Circuits and Systems (MWSCAS), College Station, USA, August 2014, pp. 362–365.
50. 50)
  - 31. Wei, Q., Liu, D., Liu, Y., et al: ‘Optimal constrained self-learning battery sequential management in microgrid via adaptive dynamic programming’, IEEE/CAA J. Autom. Sinica, 2017, 4, (2), pp. 168–176.
51. 51)
  - 22. Liang, Y., He, L., Cao, X., et al: ‘Stochastic control for smart grid users with flexible demand’, IEEE Trans. Smart Grid, 2013, 4, (4), pp. 2296–2308.
52. 52)
  - 50. IBM CPLEX: ‘IBM CPLEX homepage 2019’. Available at https://www.ibm.com/analytics/cplex-optimizer, accessed November 2019.

Stochastic home energy management system via approximate dynamic programming

References

Related content