Vehicle-mounted far-infrared pedestrian detection using multi-object tracking

Deep learning-based object detection has achieved great success on visible light images. However, it still faces various challenges in vehicle-mounted far-infrared (FIR) pedestrian detection, such as excessive false positives and false negatives due to the low contrast and small intra-class difference in FIR images, which hinder it applying to real-world. In this work, we propose a novel framework applying multi-object tracking (MOT) for detecting. Specifically, Faster R-CNN is used to obtain detected objects and multiple KCF trackers is used to track every object trajectory. Hungarian algorithm is used for bounding box matching to get historical information of trajectories to improve detection performance. To avoid KCF drift, we filter results with tracking confidence in KCF and only use the tracking bounding boxes with high confidence for matching. Besides, the KCF tracker only continues tracking when a trajectory obtains high detection score in the latest frame. Considering small intraclass difference in FIR images, IOU is combined with moving direction information for Hungarian matching. Finally, we propose a SCUT-MOT dataset to verify our method. Compared with Faster R-CNN and SORT on this dataset, our method achieves 8.9% and 2.4% precision improvement respectively.