Segmentation from localization: a weakly supervised semantic segmentation method for resegmenting CAM

Semantic segmentation has wide applications in computer vision tasks. Due to the high labor cost of pixel-level annotation, weakly supervised semantic segmentation(WSSS) methods based on image-level labels have become an important research topic. However, existing WSSS based on image-level labels has problems such as sparse segmentation results and inaccurate object boundaries. To overcome these problems, we propose a novel locate-then-segment framework that separates the localization process and segmentation process of WSSS. During the localization process we use class activation map(CAM) to locate the rough position of the object as most WSSS methods do. During the segmentation process, we focused on designing the object segmenter to refine the CAM to obtain the pseudo mask. The object segmenter consists of a dual localization feature fusion module and a boundary enhancement decoder. The former effectively extracts the semantic features of the object and finds the whole object; the latter judges long-range pixels to search for the exact object boundary. Additionally, we utilize extra pixel-level labels to train our object segmenter and add some constraints to optimize its training process. Finally, we apply the trained object segmenter to weakly supervised segmented data to improve the prediction results of CAM. Experimental results show that our proposed method significantly improves the quality of pseudo masks and obtains competitive segmentation results. Compared to existing methods, our method has the best result on the PASCAL VOC 2012 validation set with 68.8% mIoU and the competitive result on the test set with 67.9% mIoU. Our method outperforms all CNN-based methods on the MS COCO 2014 validation set, second only to transformer-based methods, achieving 36.5% mIoU. Code is available at https://github.com/wjlbnw/SegmentationFromLocalization.