Robust Deep Simple Online Real-time Tracking

Simple Online and Real-time Tracking (SORT) and its deep extension (DeepSORT) are simple, fast, and effective multi-object tracking by detection frameworks. Their main strengths are simplicity and speed. However, they still suffer from some problems, such as identity switch, instance merge, and many false positives, which prevent the tracking results from being used for subsequent tasks such as counting. In this paper, we strengthen and improve the tracking using EfficientDet and DeepSORT. In our approach, the motion prediction uses appearance, and the appearance embedding uses location. First, we modify the deep detection network to predict the objects' motion in the next frame by leveraging the attention between the current image and the next image. Second, an appearance-based metric is used to associate detection to tracks after false negatives and occlusion. This metric is a learned Mahalanobis distance between two feature descriptors constructed using EfficientDet and attention given to regions of interest from their images. Finally, we count only high confidence tracks having a minimum frequency of apparition. Our approach has been applied to a challenging real-life problem, namely seabed species tracking and counting. Our experimental results show that Robust DeepSORT reduces identity switches and merges. Thus, it improves tracking and counting evaluation measures while keeping the simplicity of the original DeepSORT.