High-Level Task-Driven Single Image Deraining: Segmentation in Rainy Days

Deraining driven by semantic segmentation task is very important for autonomous driving because rain streaks and raindrops on the car window will seriously degrade the segmentation accuracy. As a pre-processing step of semantic segmentation network, a deraining network should be capable of not only removing rain in images but also preserving semantic-aware details of derained images. However, most of the state-of-the-art deraining approaches are only optimized for high PSNR and SSIM metrics without considering objective effect for high-level vision tasks. Not only that, there is no suitable dataset for such tasks. In this paper, we first design a new deraining network that contains a semantic refinement residual network (SRRN) and a novel two-stage segmentation aware joint training method. Precisely, our training method is composed of the traditional deraining training and the semantic refinement joint training. Hence, we synthesize a new segmentation-annotated rain dataset called Raindrop-Cityscapes with rain streaks and raindrops which makes it possible to test deraining and segmentation results jointly. Our experiments on our synthetic dataset and real-world dataset show the effectiveness of our approach, which outperforms state-of-the-art methods and achieves visually better reconstruction results and sufficiently good performance on semantic segmentation task.