Siamada: visual tracking based on Siamese adaptive learning network

Recently, Siamese trackers based on region proposal networks (RPN) have gained a lot of popularity. However, the design of RPN requires manual tuning of parameters such as object-anchor intersection over union (IoU) and relative weights for different tasks, which is a difficult and expensive process for model training. To address this issue, we propose a novel Siamese adaptive learning network (SiamAda) for visual tracking, allowing the model trained in a flexible way. Rather than IoU-based anchor assignment, the proposed network uses spatial alignment and model learning status as criteria for anchor quality evaluation, and a Gaussian mixture distribution for adaptive assignment. Moreover, aiming at the inconsistency problem between classification confidence and localization accuracy, a localization branch is designed to predict the IoU for each candidate anchor box, responsible for localization quality assessment. Furthermore, to avoid the tricky relative weight tuning between each task’s loss, multi-task learning with homoscedastic uncertainty is employed to adaptively weigh these multiple losses. Extensive experiments on challenging benchmarks, namely OTB2015, VOT2018, DTB70, UAV20L, GOT-10k and LaSOT validate the superiority of our tracker. The ablation studies also illustrate the advantage of each strategy presented in this paper.