Learning to perform joint image super-resolution and rain removal via a single-convolutional neural network

In outdoor low-level vision systems, not only is the resolution of the imaging system important, but rain corrupts the visibility of outdoor scenes and may cause computer vision systems to fail. We present a deep convolutional neural network (CNN) architecture for simultaneously performing single-image super-resolution and rain removal. Instead of learning an end-to-end mapping between the low-resolution rainy images and high-resolution clean images in the original image space, we train our network in the detail space, i.e., the space obtained by high-pass filtering the original image. The proposed CNN has a lightweight structure, yet it outperforms super-resolution and rain removal consecutively by a significantly large margin (>1 dB on average). (C) 2018 SPIE and IS&T