Overinterpretation of evaluation results in machine learning studies for maize yield prediction: A systematic review

An increasing number of studies are focused on developing and evaluating machine learning (ML) models for crop yield prediction. However, poor reporting and intentional or unintentional misuse of language can distort the interpretation of evaluation results and make them appear more favorably than they actually are. The prevalence of overinterpretation of evaluation results is not yet known. To address this gap, we determined the frequency of overinterpretation in different sections of articles evaluating prediction models for maize as a specific crop. We conducted a prospectively registered systematic review, searching Scopus and Web of Science for journal articles reporting the evaluation of ML prediction models for maize yield, published from 2014 and 2023. We evaluated 207 articles based on reporting criteria adapted from overinterpretation frameworks in the biomedical literature. Overly positive words were used to describe evaluation results in 58 conclusion sections (28.0%) and 52 abstracts (25.1%). Fifty-two articles (25.1%) reported results for the coefficient of determination (R2) inconsistent between abstract and main text. Forty-eight articles (23.2%) incorrectly interpreted correlation coefficients as a measure of performance. Thirty articles (14.5%) reported wrong formulas for at least one of five standard metrics. In 25 articles (12.1%), at least one metric lacked or had an incorrect unit of measurement. In each 21 articles (10.1%), the reporting of the normalized root mean square error (NRSME) was inconsistent between abstract and main text, and R2 was wrongly interpreted as correlation. In summary, overinterpretation of evaluation results in ML studies for maize yield prediction was a frequently observed phenomenon. The findings have important implications for both the interpretation of individual studies and the broader accumulation of knowledge. Authors, reviewers, and editorial boards should put more emphasis on a complete, unambiguous, and correct reporting of evaluation results for which we submit recommendations.