access icon free Deep reinforcement learning-based optimal data-driven control of battery energy storage for power system frequency support

© The Institution of Engineering and Technology
Received 04/05/2020, Accepted 09/09/2020, Revised 28/07/2020, Published 18/11/2020

A battery energy storage system (BESS) is an effective solution to mitigate real-time power imbalance by participating in power system frequency control. However, battery aging resulted from intensive charge–discharge cycles will inevitably lead to lifetime degradation, which eventually incurs high-operating costs. This study proposes a deep reinforcement learning-based data-driven approach for optimal control of BESS for frequency support considering the battery lifetime degradation. A cost model considering battery cycle aging cost, unscheduled interchange price, and generation cost is proposed to estimate the total operational cost of BESS for power system frequency support, and an actor–critic model is designed for optimising the BESS controller performance. The effectiveness of the proposed optimal BESS control method is verified in a three-area power system.

Inspec keywords: power engineering computing; battery storage plants; optimal control; frequency control; learning (artificial intelligence); secondary cells; optimisation; power generation control

Other keywords: unscheduled interchange price; deep reinforcement learning; three-area power system; optimal control; optimal BESS control method; optimal data-driven control; BESS controller performance; actor–critic model; battery cycle aging cost; battery aging; power system frequency support; high-operating costs; total operational cost; battery lifetime degradation; power system frequency control; battery energy storage system; intensive charge–discharge cycles; data-driven approach; real-time power imbalance mitigation; generation cost

Subjects: Secondary cells; Other power stations and plants; Secondary cells; Optimal control; Power system control; Neural computing techniques; Power engineering computing; Optimisation techniques; Control of electric power systems; Optimisation techniques; Frequency control

References

    1. 1)
      • 24. Liao, K., Xu, Y.: ‘A robust load frequency control scheme for power systems based on second-order sliding mode and extended disturbance observer’, IEEE Trans. Ind. Inf., 2018, 14, (7), pp. 3076–3086.
    2. 2)
      • 10. Ju, C., Wang, P., Goel, L., et al: ‘A two-layer energy management system for microgrids with hybrid energy storage considering degradation costs’, IEEE Trans. Smart Grid, 2018, 9, (6), pp. 6047–6057.
    3. 3)
      • 7. Jinlei, S., Lei, P., Ruihang, L., et al: ‘Economic operation optimization for 2nd use batteries in battery energy storage systems’, IEEE Access, 2019, 7, pp. 41852–41859.
    4. 4)
      • 12. Liu, W., Xu, Y.: ‘A data-driven method for online health estimation of Li-Ion batteries with a novel energy-based health indicator’, IEEE Trans. Energy Convers., 2020, 35, pp. 1715–1718.
    5. 5)
      • 14. Wang, Y., Wan, C., Zhou, Z., et al: ‘Improving deployment availability of energy storage with data-driven AGC signal models’, IEEE Trans. Power Syst., 2018, 33, (4), pp. 4207–4217.
    6. 6)
      • 2. Wang, Y., Xu, Y., Tang, Y., et al: ‘Aggregated energy storage for power system frequency control: a finite-time consensus approach’, IEEE Trans. Smart Grid, 2019, 10, (4), pp. 3675–3686.
    7. 7)
      • 22. Yousef, H.A., Kharusi, K.A., Albadi, M.H., et al: ‘Load frequency control of a multi-area power system: an adaptive fuzzy logic approach’, IEEE Trans. Power Syst., 2014, 29, (4), pp. 1822–1830.
    8. 8)
      • 16. Yan, Z., Xu, Y., Wang, Y., et al: ‘Data-driven economic control of battery energy storage system considering battery degradation’. Presented at ICPES 2019, Perth, Australia, December, 2019.
    9. 9)
      • 19. He, G., Chen, X., Zhou, Y., et al: ‘Study on the price design of AGC service in China’ in ‘Applied mechanics and materials’, vol. 401 (Trans Tech Publications Ltd, Switzerland, 2013).
    10. 10)
      • 25. Lillicrap, T., Hunt, J., Pritzel, A., et al: ‘Continuous control with deep reinforcement learning’. Presented at ICML, New York, USA, 19–24 June 2016.
    11. 11)
      • 4. Masuta, T., Yokoyama, A.: ‘Supplementary load frequency control by use of a number of both electric vehicles and heat pump water heaters’, IEEE Trans. Smart Grid, 2012, 3, (3), pp. 1253–1262.
    12. 12)
      • 6. Oudalov, A., Chartouni, D., Ohler, C.: ‘Optimizing a battery energy storage system for primary frequency control’, IEEE Trans. Power Syst., 2007, 22, (3), pp. 1259–1266.
    13. 13)
      • 26. Laresgoiti, I., Käbitz, S., Ecker, M., et al: ‘Modeling mechanical degradation in lithium ion batteries during cycling: solid electrolyte interphase fracture’, J. Power Sources, 2015, 300, pp. 112–122.
    14. 14)
      • 5. Vachirasricirikul, S., Ngamroo, I.: ‘Robust LFC in a smart grid with wind power penetration by coordinated V2G control and frequency controller’, IEEE Trans. Smart Grid, 2014, 5, (1), pp. 371–380.
    15. 15)
      • 13. Bui, V., Hussain, A., Kim, H.: ‘Double deep Q-learning-based distributed operation of battery energy storage system considering uncertainties’, IEEE Trans. Smart Grid, 2020, 11, (1), pp. 457–469.
    16. 16)
      • 15. Sánchez, F., Gonzalez-Longatt, F., Rodríguez, A., et al: ‘Dynamic data-driven SoC control of BESS for provision of fast frequency response services’. Presented at 2019 IEEE PESGM, Atlanta, GA, USA, 2019.
    17. 17)
      • 3. Shim, J.W., Verbič, G., Kim, H., et al: ‘On droop control of energy-constrained battery energy storage systems for grid frequency regulation’, IEEE Access, 2019, 7, pp. 166353–166364.
    18. 18)
      • 11. Xu, B., Zhao, J., Zheng, T., et al: ‘Factoring the cycle aging cost of batteries participating in electricity markets’, IEEE Trans. Power Syst., 2018, 33, (2), pp. 2248–2259.
    19. 19)
      • 21. Bevrani, H.: ‘Robust power system frequency control’ (Springer International Publishing, Cham, Switzerland, 2014).
    20. 20)
      • 20. Chanana, S., Kumar, A.: ‘A price based automatic generation control using unscheduled interchange price signals in Indian electricity system’, Int. J. Eng., Sci. Technol.2010, 2, pp. 23–30.
    21. 21)
      • 8. Zhu, Y., Liu, C., Sun, K., et al: ‘Optimization of battery energy storage to improve power system oscillation damping’, IEEE Trans. Sustain. Energy, 2019, 10, (3), pp. 1015–1024.
    22. 22)
      • 18. Miao, Z., Fan, L.: ‘Achieving economic operation and secondary frequency regulation simultaneously through local feedback control’, IEEE Trans. Power Syst., 2017, 32, (1), pp. 85–93.
    23. 23)
      • 9. Zhang, Y.J.A., Zhao, C., Tang, W., et al: ‘Profit-maximizing planning and control of battery energy storage systems for primary frequency control’, IEEE Trans. Smart Grid, 2018, 9, (2), pp. 712–723.
    24. 24)
      • 17. Henderson, P.D., Klairnan, H., Ginnetti, J., et al: ‘Cost aspects of AGC, inadvertent energy and time error’, IEEE Trans. Power Syst., 1990, 5, (1), pp. 111–118.
    25. 25)
      • 23. Wang, Y, Xu, Y, Tang, Y.: ‘Distributed aggregation control of grid-interactive smart buildings for power system frequency support’, Appl. Energy, 2019, 251, p. 113371.
    26. 26)
      • 1. Mercier, P., Cherkaoui, R., Oudalov, A.: ‘Optimizing a battery energy storage system for frequency control application in an isolated power system’, IEEE Trans. Power Syst., 2009, 24, (3), pp. 1469–1477.
http://iet.metastore.ingenta.com/content/journals/10.1049/iet-gtd.2020.0884
Loading

Related content

content/journals/10.1049/iet-gtd.2020.0884
pub_keyword,iet_inspecKeyword,pub_concept
6
6
Loading