Dual Branch Framework Using Positive and Negative Learning for Weakly Supervised Semantic Segmentation

Weakly supervised semantic segmentation (WSSS) has received considerable interest since it relies only on image-level annotations rather than fine-grained pixel-wise annotations, which require vast human labor. Generating pseudo-masks (a.k.a. seeds) is arguably the most standard step for WSSS. The main difficulty is that seeds are usually sparse and incomplete. In this paper, we propose a dual branch framework by positive and negative learning for WSSS, which distills more accurate semantic information from multiple seeds instead of struggling to refine a single seed. First, we integrate different classificatinetworks with class activation maps to generate multiple seeds. Then, considering that richer information exists in different seeds, we perform multi-source information distillation to obtain aggregated seeds that include clean labels and noisy labels, which are more comprehensive and reliable to train a segmentation model. Furthermore, we construct a dual branch segmentation network, which makes full use of correct information while eliminating incorrect information from distilled seeds that are further acquired by aggregated seeds. When evaluated on two benchmark datasets, our method outperforms state-of-the-art methods, demonstrating the superior performance.