Visual Active Tracking Algorithm for UAV Cluster Based on Deep Reinforcement Learning

UAV cluster has been widely used in many fields, the traditional methods often rely on accurate positioning information and reliable networking method, but these methods will fail in GPS-denied or unstable zones. In this study, a novel end-to-end active tracking algorithm for UAV cluster is proposed. For this task, conventional methods tackle tracking and robotics control tasks separately, and the resulting sensing and control system is difficult to tune jointly. To address this problem, we combine the reinforcement learning agent with a deep neural network, Deep Reinforcement Learning (DRL) can extract visual features from infinite state space and directly output the tracking actions of UAV. In addition, multiple visual attention blocks are added to the agent model based on standard DRL algorithms to improve the ability of the network to quickly extract features. We use Airsim and UE4 simulator to test the algorithm, the results demonstrate that the models of dueling-DQN (Deep Q-learning Network) equipped with SE (Squeeze-and-Excitation) blocks obtain better results than others. Compared with traditional tracking algorithms, the SE-DQN method improves the tracking success rates by 15%20%. At the same time, the method can also be adopted as the formation algorithm for small UAV clusters.