SFSM: sensitive feature selection module for image semantic segmentation

One of the great challenges for image semantic segmentation is the loss of object details caused by the extensive use of convolution and pooling operations, such as blurred edges and lines, ignoring small objects, etc. To address these problems, we propose the sensitive feature selection module (SFSM), which learns the distribution characteristics of each pixel on different channels at the same location by utilizing the feature maps from prior convolution layers. Then, the obtained weights are used to reweight each pixel on different channels, so that the object boundaries and small objects can be better focused by the network in the feature extraction process. At last, the information obtained by SFSM is combined with the original features to further improve the feature representation and help to obtain more accurate segmentation results. Experimental results show that our SFSM algorithm can improve the performance of semantic segmentation networks. By integrating our SFSM into FCN and DeepLabv3, we can get 0.21% and 0.6% accuracy improvement on the PASCAL VOC 2012 dataset respectively. For the Cityscapes dataset, although the segmentation task is relatively complicated, our improved networks still achieve excellent performance. To further verify our module is not restricted to specific networks or datasets, we embed it into DoubleU-Net to do medical image segmentation task on dataset ISIC-2018 and get 0.16% accuracy improvement.