Machine learning imputation of missing Mesonet temperature observations

被引:10
|
作者
Boomgard-Zagrodnik, Joseph P. [1 ]
Brown, David J. [2 ]
机构
[1] Washington State Univ, Pullman, WA 99164 USA
[2] METER Grp, Pullman, WA USA
关键词
Machine learning; Big data; Surface weather observations; Degree day models; Missing data imputation; WEATHER DATA; QUALITY; MAXIMUM; WINDS;
D O I
10.1016/j.compag.2021.106580
中图分类号
S [农业科学];
学科分类号
09 ;
摘要
Uninterrupted and reliable weather data is a necessary foundation for agricultural decision making, required for models based on accumulated growing degree days (GDD), chill units, and evapotranspiration. When a weather station experiences a mechanical or communications failure, a replacement (imputed) value should be substituted for any missing data. This study introduces a machine learning, network-based approach to imputing missing 15-minute and daily maximum/minimum air temperature observations from 8.5 years of air temperature, relative humidity, wind, and solar radiation observations at 134 AgWeatherNet (AWN) stations in Washington State. A random forest imputation model trained on temperature and humidity observations from the full network predicted 15-minute, daily maximum, and daily minimum temperature values with mean absolute errors of 0.43 degrees C, 0.53 degrees C, and 0.70 degrees C, respectively. Sensitivity experiments determined that imputation skill was related a number of external factors including volume and type of training data, proximity of surrounding stations, and regional topography. In particular, nocturnal cold air flows in the upper Yakima Valley of southcentral Washington caused temperature to be less correlated with surrounding stations in the overnight hours. In a separate experiment, the imputation model was used to predict base- 10 degrees C GDD on 2020 July 1 trained entirely on 15-minute station data from previous years. Even with the entire season of observations missing, the model predicted the GDD value within an average error 1.4% with 125 of 134 stations within 5% of observations. Since missing data can typically be resolved within a timeframe of a few days, the network-based imputation model is a sufficient substitute for short periods of missing observational weather data. Other potential applications of an imputation model are briefly discussed.
引用
收藏
页数:11
相关论文
共 50 条
  • [1] Analysis of Machine Learning Based Imputation of Missing Data
    Rizvi, Syed Tahir Hussain
    Latif, Muhammad Yasir
    Amin, Muhammad Saad
    Telmoudi, Achraf Jabeur
    Shah, Nasir Ali
    CYBERNETICS AND SYSTEMS, 2023,
  • [2] Approximate Imputation Method for Missing Data in Machine Learning
    Cao W.
    Chu Y.
    Li X.
    1600, Xi'an Jiaotong University (51): : 142 - 148
  • [3] Missing Data Imputation using Machine Learning Algorithm for Supervised Learning
    Cenitta, D.
    Arjunan, R. Vijaya
    Prema, K., V
    2021 INTERNATIONAL CONFERENCE ON COMPUTER COMMUNICATION AND INFORMATICS (ICCCI), 2021,
  • [4] Missing value imputation using unsupervised machine learning techniques
    Raja, P. S.
    Thangavel, K.
    SOFT COMPUTING, 2020, 24 (06) : 4361 - 4392
  • [5] Missing value imputation using unsupervised machine learning techniques
    P. S. Raja
    K. Thangavel
    Soft Computing, 2020, 24 : 4361 - 4392
  • [6] Machine Learning Based Missing Data Imputation in Categorical Datasets
    Ishaq, Muhammad
    Zahir, Sana
    Iftikhar, Laila
    Bulbul, Mohammad Farhad
    Rho, Seungmin
    Lee, Mi Young
    IEEE ACCESS, 2024, 12 : 88332 - 88344
  • [7] ExtraImpute: A Novel Machine Learning Method for Missing Data Imputation
    Alabadla, Mustafa
    Sidi, Fatimah
    Ishak, Iskandar
    Ibrahim, Hamidah
    Affendey, Lilly Suriani
    Hamdan, Hazlina
    JOURNAL OF ADVANCES IN INFORMATION TECHNOLOGY, 2022, 13 (05) : 470 - 476
  • [8] Performance Analysis of Machine Learning Algorithms for Missing Value Imputation
    Abidin, Nadzurah Zainal
    Ismail, Amelia Ritahani
    Emran, Nurul A.
    INTERNATIONAL JOURNAL OF ADVANCED COMPUTER SCIENCE AND APPLICATIONS, 2018, 9 (06) : 442 - 447
  • [9] Machine Learning Aids Imputation of Missing Petrophysical Data in Iraqi Reservoir
    Abdulkhaleq, Hussein B.
    Al-Mudhafar, Watheq J.
    Wood, David A.
    JPT, Journal of Petroleum Technology, 1600, 76 (08): : 58 - 61
  • [10] Sharpening the BLADE: Missing Data Imputation Using Supervised Machine Learning
    Suresh, Marcus
    Taib, Ronnie
    Zhao, Yanchang
    Jin, Warren
    AI 2019: ADVANCES IN ARTIFICIAL INTELLIGENCE, 2019, 11919 : 215 - 227