Your browser does not support JavaScript!
http://iet.metastore.ingenta.com
1887

access icon openaccess Reinforcement learning for optimal policy learning in condition-based maintenance

This is an open access article published by the IET under the Creative Commons Attribution-NonCommercial-NoDerivs License (http://creativecommons.org/licenses/by-nc-nd/3.0/)
Received 29/02/2020, Accepted 04/09/2020, Revised 12/08/2020, Published 09/09/2020

Condition-based maintenance (CBM) involves taking decisions on maintenance or repair based on the actual deterioration conditions of the components. The long-run average cost is minimised by choosing the right maintenance action at the right time. In this study, the CBM decision-making problem is modelled as a continuous semi-Markov decision process (CSMDP). It consists of a chain of states representing various stages of deterioration, a set of maintenance actions, their costs and scheduled inspection policy. The application of a reinforcement learning (RL) algorithm based on the average reward for CSMDPs in CBM is described. The RL algorithm is used to learn the optimal maintenance decisions and inspection schedule based on the current health state of the component.

Inspec keywords: cost reduction; maintenance engineering; Markov processes; inspection; condition monitoring; optimisation; scheduling; decision making; decision theory

Other keywords: reinforcement learning; optimal maintenance decisions; inspection schedule; average cost; CBM decision-making problem; continuous semiMarkov decision process; maintenance action; component health state; CSMDP; condition-based maintenance; deterioration conditions; optimal policy learning; inspection policy; RL algorithm

Subjects: Markov processes; Optimisation; Financial management; Inspection and quality control; Production management; Maintenance and reliability; Optimisation techniques; Statistics

References

    1. 1)
      • 3. Yousefi, N., Tsianikas, S., Coit, D.W.: ‘Reinforcement learning for dynamic condition-based maintenance of a system with individually repairable components’, Qual. Eng., 2020, 32, pp. 388–408.
    2. 2)
      • 7. Zhang, N., Si, W.: ‘Deep reinforcement learning for condition-based maintenance planning of multi-component systems under dependent competing risks’, Reliab. Eng. Syst. Safety, 2020, 203, p. 107094.
    3. 3)
      • 4. Huang, J., Chang, Q., Arinez, J.: ‘Deep reinforcement learning based preventive maintenance policy for serial production lines’, Expert Syst. Appl., 2020, 10, p. 100034.
    4. 4)
      • 9. Rocchetta, R., Bellani, L., Compareb, M., et al: ‘A reinforcement learning framework for optimal operation and maintenance of power grids’, Appl. Energy, 2019, 241, pp. 291–301.
    5. 5)
      • 16. Amari, S., McLaughlin, L., Pham, H.: ‘Cost-effective condition-based maintenance using Markov decision processes’. Proc. of Annual Reliability and Maintainability Symp., Newport Beach, CA, USA, 2006, pp. 464–469.
    6. 6)
      • 15. Ling, Z., Wang, X., Qu, F.: ‘Reinforcement learning-based maintenance scheduling for resource constrained flow line system’. IEEE 4th Int. Conf. on Control Science and System Engineering, Wuhan, China, 2018.
    7. 7)
      • 2. Knowles, M., Baglee, D., Wermter, S.: ‘Reinforcement learning for scheduling of maintenance’, in Bramer, M., Petridis, M., Hopgood, A. (Eds.): ‘Research and development in intelligent systems XXVII’ (Springer, London, UK, 2010).
    8. 8)
      • 1. Mahadevan, S., Marchalleck, N., Das, T.K., et al: ‘Self-improving factory simulation using continuous-time average-reward reinforcement learning’. Proc. of Fourteenth Int. Conf. Machine Learning, Nashville, Tennessee, USA, 1997.
    9. 9)
      • 5. Xanthopoulos, A.S., Kiatipis, A., Koulouriotis, D.E., et al: ‘Reinforcement learning-based and parametric production-maintenance control policies for a deteriorating manufacturing system’, IEEE Access, 2018, 6, pp. 576–588.
    10. 10)
      • 11. Mahadevan, S., Theocharous, G.: ‘Optimizing production manufacturing using reinforcement learning’. Proc. of Eleventh FLAIRS Conf., Florida, USA, 1998.
    11. 11)
      • 13. Chen, D., Trivedi, K.: ‘Optimization for condition-based maintenance with semi-Markov decision process’, Reliab. Eng. Syst. Saf., 2005, 90, pp. 25–29.
    12. 12)
      • 8. Golmakani, H.R., Fattahipour, F.: ‘Optimal replacement policy and inspection interval for condition-based maintenance’, Int. J. Prod. Res., 2011, 49, (17), pp. 5153–5167.
    13. 13)
      • 12. Wang, G., Mahadevan, S.: ‘A greedy divide-and-conquer approach to optimizing large manufacturing systems using reinforcement learning’, 1998.
    14. 14)
      • 10. Wei, S., Bao, Y., Li, H.: ‘Optimal policy for structure maintenance: a deep reinforcement learning framework’, Struct. Saf., 2020, 83, p. 101906.
    15. 15)
      • 6. Lam, J.Y.J., Banjevic, D.: ‘A myopic policy for optimal inspection scheduling for condition-based maintenance’, Reliab. Eng. Syst. Saf., 2015, 144, pp. 1–11.
    16. 16)
      • 14. Marseguerra, M., Zio, E., Podofillini, L.: ‘Condition based maintenance optimization by means of genetic algorithms and Monte Carlo simulation’, Reliab. Eng. Syst. Saf., 2002, 77, pp. 151–166.
http://iet.metastore.ingenta.com/content/journals/10.1049/iet-cim.2020.0022
Loading

Related content

content/journals/10.1049/iet-cim.2020.0022
pub_keyword,iet_inspecKeyword,pub_concept
6
6
Loading
Print this page
This is a required field
Please enter a valid email address