Attentional and adversarial feature mimic for efficient object detection

In this paper, we focus on learning efficient object detectors by knowledge (or network) distillation. More specifically, we mimic features from deeper and larger teacher networks to help train better efficient student networks. Unlike the previous method that mimics features through minimizing an L2 loss between feature generated by teacher and student networks, we propose an attentional and adversarial feature mimic (AAFM) method which consists of an attentional feature mimic module and an adversarial feature mimic module, where the former module uses an attentional L2 loss which learns to pay attention to important object-related regions for feature mimic, and the latter module uses an adversarial loss which makes features generated by teacher and student networks have similar distributions. We apply our AAFM method in the two-stage Faster R-CNN detector. Experiments on the PASCAL VOC 2007 and COCO datasets show that our method consistently improves the performance of detectors without feature mimic or with other feature mimic methods. In particular, our method obtains 72.1% mAP on the PASCAL VOC 2007 dataset using the ResNet-18-based detector.