Deterministic and fuzzy verification methods for a hierarchy of numerical models

Deterministic and fuzzy verification methods are compared to assess the Quantitative Precipitation Forecasts (QPF) performance of an hierarchy of models run at Meteo-France: two operational models (ARPEGE and ALADIN) and a prototype version of the high resolution model AROME. The reference data are 24-h accumulated rainfall values measured by the French climatological network of rain gauges. The deterministic forecasts are converted to frequencies of threshold exceedence in a neighbourhood in order to apply a fuzzy verification method, able to determine the influence of the double penalty. Local and regional versions of the Brier skill score (BSS) are computed and persistence is the selected reference system. An optimal size of the neighbourhood can be determined from the local version of the BSS. The regional version of the BSS increases with the size of the neighbourhood and provides useful information oil the reduction of the error for large scales. All the scores show that the ALADIN 3DVAR implementation improves the quality of the ALADIN QPF in comparison to the ARPEGE QPF. Deterministic verification methods show that AROME improves the QPF for weak precipitation but produces too many false alarms for heavy rain. Moreover, the fuzzy approach removes more substantially the double penalty for the heavy rain forecasts of AROME than for the light precipitation of ALADIN. After recalibration the ALADIN QPF have better deterministic scores than the AROME QPF. Nevertheless, fuzzy verification methods prove the contrary: for scales larger than 50 km, the AROME scores exceed the ALADIN scores, but not significantly. Copyright (C) 2008 Royal Meteorological Society