Weakly Supervised Learning of Instance Segmentation with Confidence Feedback

Only provided with image category supervision, weakly supervised instance segmentation is a challenging yet significant task when required to simultaneously learn object locations and instance segmentation. In this paper, we propose a region-based multibranch network with feed-forward and feedback procedures to estimate image classifiers, object detectors, and mask predictors in an end-to-end manner. In the feed-forward procedure, the instance confidences estimated by the image classification branch are transferred to train object detectors and mask predictors. In the feedback procedure, the instance confidences retrieved from detection outputs and mask confidences obtained from segmentation outputs are employed for classifier enhancement. With iterative feed-forward and feedback procedures, our approach produces a closed-loop multitask learning mechanism that can efficiently correct object localization while generating high-quality instance segmentation. On PASCAL VOC benchmark datasets, our methods greatly improve the performance of weakly supervised object detection (WSOD) and weakly supervised instance segmentation (WSIS).