Accurate estimation of global horizontal irradiance from Himawari-8 satellite data using interpretable hybrid models that reduce shortcut learning

Accurate near-real-time estimates of global horizontal irradiance (GHI) are crucial for understanding the Earth's energy budget and assessing the potential of solar energy as a renewable resource for mitigating climate change. However, dynamic atmospheric variations, such as aerosols and clouds, pose a challenge to accurate GHI estimation due to their unpredictable nature, leading to discrepancies between observed and simulated values, especially over short timeframes. To address this challenge, we combine the band information from the Himawari-8 satellite to enhance the ability to capture specific information such as clouds, aerosols, and atmospheric conditions. We further employ feature selection methods to input only the selected informative features into a hybrid model to mitigate possible shortcut learning. Our results show that the hybrid model (R-2=0.90, RMSE = 88.38 W/m(2), RPD = 3.10, MAE = 57.75 W/m(2)) not only outperforms the deep neural network (DNN) (R-2=0.82, RMSE = 116.78 W/m(2), RPD = 2.35, MAE = 80.47 W/m(2)) and the long short-term memory network (LSTM) (R-2=0.87, RMSE = 97.94 W/m(2), RPD = 2.80, MAE = 65.05 W/m(2)) in terms of performance, but also exhibits good adaptability under different weather conditions. The results suggest that the use of a hybrid model helps balance the importance of various features, thereby reducing shortcut learning. This leads to more accurate and reliable estimates of GHI and provides new insights for obtaining high-frequency GHI over large spatial scales.