GAN-based data augmentation for semantic segmentation in multiple weathers

Datasets play an important role in determining the features that deep neural networks can acquire, but they can also contain unintended biases when constructing datasets. The BDD100K dataset, famous for its semantic segmentation task, was collected to include traffic scenes for multiple weather conditions. However, due to differences in frequency of occurrence, there is a bias in the number of data for each weather condition. Therefore, the segmentation network trained by BDD100K has poor recognition performance in some weather conditions. Semantic segmentation is an urgent issue because it is expected to be applied to traffic scene recognition systems. In this paper, we aim to improve the performance of semantic segmentation by designing a method that generates images of desired weather conditions and uses them for data augmentation. In our experiments, we first show that the image generation method we have developed produces images of a quality that can be used for data augmentation. Next, we examine the effect of data augmentation on the semantic segmentation task. As a result, compared to baseline, the mean intersection over union (mloU) improved by about 15% in wet weather, about 9% at night, and about 1% overall. © 2021 Japan Society for Precision Engineering. All rights reserved.