End-to-End Single Shot Detector Using Graph-Based Learnable Duplicate Removal

Non-Maximum Suppression (NMS) is widely used to remove duplicates in object detection. In strong disagreement with the deep learning paradigm, NMS often remains as the only heuristic step. Learning NMS methods have been proposed that are either designed for Faster-RCNN or rely on separate networks. In contrast, learning NMS for SSD models is not well investigated. In this paper, we show that even a very simple rescoring network can be trained end-to-end with an underlying SSD model to solve the duplicate removal problem efficiently. For this, detection scores and boxes are refined from image features by modeling relations between detections in a Graph Neural Network (GNN). Our approach is applicable to the large number of object proposals in SSD using a pre-filtering head. It can easily be employed in arbitrary SSD-like models with weight-shared box predictor. Experiments on MS-COCO and KITTI show that our method improves accuracy compared with other duplicate removal methods at significantly lower inference time.