EVALUATION OF MODEL PERFORMANCE WHEN THE OBSERVED DATA ARE SUBJECT TO ERROR

In many cases of model evaluation in physical geography, the observed data to which model predictions are compared may not be error free. This paper addresses the effect of observational errors on the mean squared error, the mean bias error and the mean absolute deviation through the derivation of a statistical framework and Monte Carlo simulation. The effect of bias in the observed values may either decrease or increase the expected values of the mean squared error and mean bias error, depending on whether model and observational biases have the same or opposite signs, respectively. Random errors in observed data tend to inflate the mean squared error and the mean absolute deviation, and also increase the variability of all the error indices considered here. The statistical framework is applied to a real example, in which sampling variability of the observed data appears to account for most of the difference between observed and predicted values. Examination of scaled differences between modelled and observed values, where the differences are divided by the estimated standard errors of the observed values, is suggested as a diagnostic tool for determining whether random observational errors are significant.