Could a perfect model ever satisfy a naive forecaster?: On grid box mean versus point verification

Numerical models forecast and box means, whereas forecasters have to deliver point forecasts, often in the form of the most severe weather possible in a small area'. There can be substantial differences between area means and point values, depending on the subgrid-scale variability. Forecasters are generally well aware of subgrid-scale variability when making local predictions of severe events. However, in evaluating the model this is often left out of consideration. An experiment was performed to estimate the effect of 'imperfectly verifying' grid box means against point observations. A 'perfect model' was constructed by taking the grid box average of high-density observations of daily precipitation amount, but the verification of this 'perfect' forecast is far from being accurate. The model deviation from the observations is minimal for small precipitation amounts, and becomes larger as more severe events are to be forecast. The error solely due to the wrong kind of verification is of the order of the forecast error itself. If one has to verify against point observations, the baseline to compare numerical models against is not the theoretical best score Value but the score achieved by the "perfect model". Thus, this study underlines again the imperative to compare numerical models against upscaled observation. Even a perfect model could not nearly satisfy a naive forecaster who Just wants to pass a model Value to a Customer in need of a point forecast. Good forecasters apply some post-processing in order to estimate the subgrid-scale variability and thus the probability of exceeding some threshold. Copyright (c) 2008 Royal Meteorological Society.