Deep Convolutional Encoder-Decoder Network with Model Uncertainty for Semantic Segmentation

We propose a new semantic segmentation method and the necessity of certainty for practical use of semantic segmentation in scene understanding. We implement a deep fully convolutional encoder-decoder neural network for semantic segmentation. This network architecture makes the segmentation accuracy improve by retaining boundary details in the extracted image representation. This accuracy means how much the segmentation results match to ground truth labels. However, the conventional evaluation method ignores unlabeled regions in ground truth labels. In other words, the segmentation results has not been evaluated in the regions of unknown objects. Toward practical use of the semantic segmentation, the evaluation should consider such regions. So it is necessary to recognize accurately whether the object is known or not. We call this factor certainty. Bayesian SegNet makes it possible to produce an uncertainty of the segmentation results with a measure of model uncertainty from the sampling of the posterior distribution of the model using Dropout. However, the uncertainty is not used for segmentation itself, and all pixels are classified into one of the predefined classes in this segmentation result. It means that the pixels within the regions of unknown objects are definitely misclassified as one of the predefined classes. Our study aims the improvement of certainty for semantic segmentation in road scene understanding with model uncertainty. Our method rejects the uncertain region and classifies it as an unknown object using the model uncertainty. We achieved improvement of certainty by our method as shown in the evaluation results. Furthermore, we indicated the possibility of the performance improvement on the deep convolutional encoder-decoder network architecture from the comparison of our network architecture with Bayesian SegNet architecture.