A Machine Learning-based Cloud Detection Algorithm for the Himawari-8 Spectral Image

Cloud Masking is one of the most essential products for satellite remote sensing and downstream applications. This study develops machine learning-based (ML-based) cloud detection algorithms using spectral observations for the Advanced Himawari Imager (AHI) onboard the Himawari-8 geostationary satellite. Collocated active observations from Cloud-Aerosol Lidar with Orthogonal Polarization (CALIOP) are used to provide reference labels for model development and validation. We introduce both daytime and nighttime algorithms that differ according to whether solar band observations are included, and the artificial neural network (ANN) and random forest (RF) techniques are adopted for comparison. To eliminate the influences of surface conditions on cloud detection, we introduce three models with different treatments of the surface. Instead of developing independent ML-based algorithms, we add surface variables in a binary way that enhances the ML-based algorithm accuracy by similar to 5%. Validated against CALIOP observations, we find that our daytime RF-based algorithm outperforms the AHI operational algorithm by improving the accuracy of cloudy pixel detection by similar to 5%, while at the same time, reducing misjudgment by similar to 3%. The nighttime model with only infrared observations is also slightly better than the AHI operational product but may tend to overestimate cloudy pixels. Overall, our ML-based algorithms can serve as a reliable method to provide cloud mask results for both daytime and nighttime AHI observations. We furthermore suggest treating the surface with a set of independent variables for future ML-based algorithm development.