Siamese-based trackers have achieved strong performance in single-target tracking. Effective feature response maps are fundamental to improving tracker performance when dealing with challenging scenes. However, most Siamese-based trackers have constant template features when tracking. This approach greatly limits the effectiveness of the tracker in complex scenes. To solve this issue, we proposed a novel tracking framework, termed as SiamATU, which adaptively performs update of template features. This update method uses a multi-stage training strategy during the training process so that the template update is gradually optimized. In addition, we designed a feature enhancement module to enhance the discriminative and robustness of the features, which focuses on the rich correlation between the template image and the search image, and then makes the model more focused on the tracking object to achieve more precise tracking. Through extensive experiments on GOT-10K, UAV123, OTB100, and other datasets, SiamATU has a leading performance, which runs at 26.23FPS, exceeding the real-time level of 25FPS.