Automatic quality control of weather data for timely decisions in agriculture

Weather data from automated stations installed in rural areas are crucial to plan agricultural operations. Yet, they are prone to measurement errors, which can result in poor planning of these operations and cause a negative impact on the environment and economic losses for the farmers. Given the increasing volumes of weather data recorded by the automatic stations, algorithms are required to detect the implausible values and help ensure the quality of that data. The goal of this research was to propose an automatic quality control method, designed with the agricultural context in mind, for eight weather variables. Air temperature, relative humidity, wind speed, global radiation, rainfall, leaf wetness duration, soil temperature and air temperature in the grass were measured at minute and hourly time step in a Belgian network of twenty-eight automated weather stations. The developed automatic checks verified missing data, range, temporal consistency, spatial consistency and internal consistency. New specific checks were developed, especially for the detection of partially clogged rain gauges, implausible series of zero in wind speed sequences, implausible combs in minute temperature data, saturation of relative humidity at a too low level and implausible leaf wetness duration. In the design of the checks, a particular attention was paid to a quick detection of the errors, as agricultural activities rely on near real-time observations. To evaluate the quality control performances, an original quantitative method was proposed, complemented with study cases. The automatic quality control performed well for all the weather variables. The algorithm was able to detect implausible values that were missed by the human operators. Performing checks on data at minute time step enabled the detection of errors that were not spotted at hourly time step. Depending on the weather variable, the checks detected between 92.6% and 100 % of the implausible values, but they raised false alarms with rates ranging between 2.7 % and 33.3 %, depending on the weather variable. It implies the need of a human supervision on the data flagged by the automatic system to avoid deleting, for instance, extreme but plausible values. Further research directions include reducing the false alarm rates and designing a robust check to differentiate snow melting in the rain gauge from implausible rains.