Condition-based maintenance for multi-state systems with prognostic and deep reinforcement learning

The utilization of prognostic information in practical engineering is increasing with the development of technology and predictive modeling. Current research on maintenance strategies for complex multi-state systems often neglects prognostic information or assumes complete availability of all component information. This paper investigates the joint maintenance strategies based on condition-based maintenance for complex multi-state systems, in which the predicted remaining useful life of some components is known. Firstly, a maintenance strategy framework is developed and the joint maintenance strategy is proposed for the studied problem. Then the deterioration process of the component, the imperfect maintenance, and prediction error models are constructed. The optimization problem is modeled as a Markov Decision Process to minimize the maintenance cost, and the system reliability constraints are established by using the universal generating function method. In addition, a deep Q-network is designed to solve the optimal maintenance policy. Finally, the traction system of a metro train is taken as an example to verify the applicability of the model and algorithm. The results show that the proposed maintenance strategy reduces the maintenance cost compared to the current maintenance strategy for both fixed maintenance intervals and dynamic maintenance intervals.