Guided probabilistic reinforcement learning for sampling-efficient maintenance scheduling of multi-component system

In recent years, multi-agent deep reinforcement learning has progressed rapidly as re-flected by its increasing adoptions in industrial applications. This paper proposes a Guided Probabilistic Reinforcement Learning (Guided-PRL) model to tackle maintenance schedul-ing of multi-component systems in the presence of uncertainty with the goal of minimiz-ing the overall life-cycle cost. The proposed Guided-PRL is deeply rooted in the Actor-Critic (AC) scheme. Since traditional AC falls short in sampling efficiency and suffers from getting stuck in local minima in the context of multi-agent reinforcement learning, it is thus chal-lenging for the actor network to converge to a solution of desirable quality even when the critic network is properly configured. To address these issues, we develop a generic frame-work to facilitate effective training of the actor network, and the framework consists of environmental reward modeling, degradation formulation, state representation, and policy optimization. The convergence speed of the actor network is significantly improved with a guided sampling scheme for environment exploration by exploiting rules-based domain expert policies. To handle data scarcity, the environmental modeling and policy optimiza-tion are approximated with Bayesian models for effective uncertainty quantification. The Guided-PRL model is evaluated using the simulations of a 12-component system as well as GE90 and CFM56 engines. Compared with four alternative deep reinforcement learn-ing schemes, the Guided-PRL lowers life-cycle cost by 34. 92% to 88. 07% . In comparison with rules-based expert policies, the Guided-PRL decreases the life-cycle cost by 23. 26% to 51. 36% .(c) 2023 Elsevier Inc. All rights reserved.