Particle swarm optimization based on temporal-difference learning for solving multi-objective optimization problems

Multi-objective evolutionary algorithms have become the most important method to deal with multi-objective optimization problems (MOP). To improve the performance of particle swarm optimization (PSO) in addressing MOPs, a multi-objective PSO based on temporal-difference learning (TDLMOPSO) is proposed in this paper. The iteration process of TDLMOPSO is transformed into a Markov decision process, particles are treated as agents, each agent has a personal archive, the states are designed for the connection of actions, the actions of particles contain all necessary behavior of them: basic movement, jump out of local optimum, and local search, and the rewards depend on the relationship between particles' positions and their personal archives. Besides, the external archive deletion strategy and the leader selection strategy are redesigned based on the unsupervised learning algorithm to enhance the diversity of solutions in the external archive. The effectiveness of TDLMOPSO is verified by applying it with other seven advanced multi-objective algorithms in MOP benchmark test suites. Furthermore, the time complexity and parameter sensitivity of TDLMOPSO are analyzed.