Multi-objective optimization-assisted single-objective differential evolution by reinforcement learning

"Learning to optimize"design systems for evolutionary algorithm (EA) automatic design have become a trend, especially for differential evolution (DE). "Learning to optimize"design systems for EAs have two main parts: an excellent "backbone"algorithm with learnable components, and a learning scheme to determine the components of the "backbone"algorithm. A good "backbone"algorithm is of great importance for the algorithm design, because it determines the algorithm design space and potential. The learning scheme determines whether we can realize the potential or not. Existing studies generally choose one developed EA as the "backbone"algorithm, which constrains the potential of the design system because the "backbone"algorithm is relatively simple. To solve the problem and design a good EA, in this paper, we first propose a three-stage hybrid DE framework for single objective optimization, called SMS-DE, which implements single- objective DE, multi-objective DE, and single-objective DE sequentially. The multi-objective DE aims to enhance exploration ability. Second, we apply the framework to two advanced DEs, JADE and LSHADE, which results in two new algorithms: SMS-JADE and SMS-LSHADE. Third, the newly proposed algorithm, SMS-LSHADE, is considered the "backbone"algorithm, and the reinforcement learning method (Q-learning) is used to control the parameter for allocating computational resources to each stage, which results in another algorithm called QSMS-LSHADE. Experimental results on the CEC 2018 test suite show that SMS-DE, SMS-JADE, and SMSLSHADE can perform significantly better than their counterparts and that SMS-QLSHADE performs the best among many developed DEs.