Semantic Foggy Scene Understanding with Synthetic Data

被引:7
作者
Christos Sakaridis
Dengxin Dai
Luc Van Gool
机构
[1] ETH Zürich,
[2] KU Leuven,undefined
来源
International Journal of Computer Vision | 2018年 / 126卷
关键词
Foggy scene understanding; Semantic segmentation; Object detection; Depth denoising and completion; Dehazing; Transfer learning;
D O I
暂无
中图分类号
学科分类号
摘要
This work addresses the problem of semantic foggy scene understanding (SFSU). Although extensive research has been performed on image dehazing and on semantic scene understanding with clear-weather images, little attention has been paid to SFSU. Due to the difficulty of collecting and annotating foggy images, we choose to generate synthetic fog on real images that depict clear-weather outdoor scenes, and then leverage these partially synthetic data for SFSU by employing state-of-the-art convolutional neural networks (CNN). In particular, a complete pipeline to add synthetic fog to real, clear-weather images using incomplete depth information is developed. We apply our fog synthesis on the Cityscapes dataset and generate Foggy Cityscapes with 20,550 images. SFSU is tackled in two ways: (1) with typical supervised learning, and (2) with a novel type of semi-supervised learning, which combines (1) with an unsupervised supervision transfer from clear-weather images to their synthetic foggy counterparts. In addition, we carefully study the usefulness of image dehazing for SFSU. For evaluation, we present Foggy Driving, a dataset with 101 real-world images depicting foggy driving scenes, which come with ground truth annotations for semantic segmentation and object detection. Extensive experiments show that (1) supervised learning with our synthetic data significantly improves the performance of state-of-the-art CNN for SFSU on Foggy Driving; (2) our semi-supervised learning strategy further improves performance; and (3) image dehazing marginally advances SFSU with our learning strategy. The datasets, models and code are made publicly available.
引用
收藏
页码:973 / 992
页数:19
相关论文
共 104 条
  • [1] Achanta R(2012)SLIC superpixels compared to state-of-the-art superpixel methods IEEE Transactions on Pattern Analysis and Machine Intelligence 34 2274-2282
  • [2] Shaji A(2014)Recent progress in road and lane detection: A survey Machine Vision and Applications 25 727-745
  • [3] Smith K(2011)A review of computer vision techniques for the analysis of urban traffic IEEE Transactions on Intelligent Transportation Systems 12 920-939
  • [4] Lucchi A(2018)DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs IEEE Transactions on Pattern Analysis and Machine Intelligence 40 834-848
  • [5] Fua P(2008)An innovative artificial fog production device improved in the European project FOG Atmospheric Research 87 242-251
  • [6] Süsstrunk S(2002)Mean shift: A robust approach toward feature space analysis IEEE Transactions on Pattern Analysis and Machine Intelligence 24 603-619
  • [7] Bar Hillel A(2011)Satellite image classification via two-layer sparse coding with biased image representation IEEE Geoscience and Remote Sensing Letters 8 173-176
  • [8] Lerner R(2010)The PASCAL visual object classes (VOC) challenge IJCV 88 303-338
  • [9] Levi D(2008)Single image dehazing ACM Transactions on Graphics (TOG) 27 72-320
  • [10] Raz G(2014)Dehazing using color-lines ACM Transactions on Graphics (TOG) 34 13-20