Contrastive Proposal Extension With LSTM Network for Weakly Supervised Object Detection

Weakly supervised object detection (WSOD) has attracted more and more attention since it only uses image-level labels and can save huge annotation costs. Most of the WSOD methods use Multiple Instance Learning (MIL) as their basic framework, which regard it as an instance classification problem. However, these methods based on MIL tend to only converge on the most discriminative regions of different instances, rather than their corresponding complete regions, that is, insufficient integrity. Inspired by the human habit of observing things, we propose a new method by comparing the initial proposals and the extended ones to optimize those initial proposals. Specifically, we propose one new strategy for WSOD by involving contrastive proposal extension (CPE), which consists of multiple directional contrastive proposal extensions (D-CPEs), and each D-CPE contains LSTM-based encoders and dual-stream decoders. Firstly, the boundary of initial proposals in MIL is extended to different positions according to well-designed sequential order. Then, the CPE compares the extended proposal and the initial one by extracting the feature semantics of them using the encoders, and calculates the integrity of the initial proposal to optimize its score. These contrastive contextual semantics will guide the basic WSOD to suppress bad proposals and improve the scores of good ones. In addition, a simple dual-stream network is designed as the decoder to constrain the temporal coding of LSTM and improve the performance of WSOD further. Experiments on PASCAL VOC 2007, VOC 2012 and MS-COCO datasets show that our method has achieved the state-of-the-art results.