On the contextual aspects of using deep convolutional neural network for semantic image segmentation

The deep convolutional neural network (CNN) has recently shown state-of-the-art performance in many image processing tasks. We examine the use of deep CNN for semantic image segmentation, which separates an input image into multiple regions corresponding to predefined object classes. We follow the most successful deep CNN-based semantic segmentation in recent years and focus on the study of the contextual aspects. To examine the context-awareness, we manually modify the context of the input images and study the effects on the segmentation results. The experiments through systematic context changes show that the model is sensitive to some context changes. We then propose context-changing data augmentation to train context-insensitive models using images solely from the original context. We experimentally validate the effectiveness of the proposed method and summarize its limitations. Finally, we discuss the need of training context-free single-class semantic segmentation models and suggest approaches for it. (C) 2018 SPIE and IS&T