Method for cloud removal of optical remote sensing images using improved CGAN network

The optical remote sensing images used in agricultural production are often affected by clouds during the acquisition process. The sharpness of acquired images can also be reduced, and thereby the decreased clarity of image makes difficult to interpret feature information. This process has posed a great challenge on subsequent applications in agricultural production, such as crop growth detection, crop classification and yield prediction. In this study, a method for cloud removal was proposed using the improved Conditional Generative Adversarial Net-work (CGAN), in order to enhance the detecting resolution of remote sensing images. A mapping relationship between pixels of the cloud and cloudless data was established by training the CGAN. The transformation of remote sensing image can be completed from cloud to cloudless on this basis. Eventually, the cloud component can be effectively removed from the optical remote sensing images. This method can also realize the data restoration of some details in optical remote sensing images during imaging processing. Therefore, a general system of network structure can be used to remove thin and thick clouds in optical remote sensing images by using the improved model. The modified generator of network can be utilized to enhance the image quality of generated data, particularly resulted from the single feature extraction of the original CGAN. The specific procedure of image processing can be: firstly, a series of convolutions were used to extract the feature information of the input images. Then, the multi-scale feature maps were obtained from the feature information using the spatial pyramid pooling operation. Finally, these feature maps with different size were restored to the original size, and thereby mixed together to generate the final cloudless optical remote sensing images. The scale of feature extraction by the generator can significantly increase in this method. Accordingly, the resulting effect on the resolution of images can also increase significantly. In order to verify the cloud removal method, some experiments were performed using the same optical remote sensing image data sets. Three types of methods were selected to compare, including the original CGAN, traditional cloud removal algorithm, and the Pix2Pix method in deep learning. Two indicators, including the Peak Signal-to-Noise Ratio (PSNR) and the Structural SIMilarity (SSIM), were introduced to make a quantitative assessment of experimental results for better evaluation. The experimental results show that: 1) the proposed method can be applied to the removal of thin and thick cloud in optical remote sensing images, indicating high resolution in both types of cloud removal. 2) Compared with the original CGAN, the quality of generated cloudless image was similar to that of the real cloudless remote sensing image. In the removal of thin cloud using the improved CGAN model, the PSNR increased by 1.64 dB, and the SSIM value increased by 0.03, whereas in the thick cloud, the PSNR increased by 1.05 dB, and the SSIM value increased by 0.04. 3) Compared with the traditional cloud removal method, the improved CGAN model removed cloud layer more thoroughly in optical remote sensing image, indicating much more realistic color of the features in the optical remote sensing image. Compared with the Pix2Pix method, some details were better recovered, particularly the landscape in the generated cloudless optical remote sensing images. The PSNR and SSIM index values of remote sensing images have been improved accordingly. These results can prove that the improved CGAN is suitable to remove clouds from the optical remote sensing images. The findings can provide an insightful idea and promising method for the remote sensing image processing in modern agriculture. © 2020, Editorial Department of the Transactions of the Chinese Society of Agricultural Engineering. All right reserved.