Unbiased calculation, evaluation, and calibration of ensemble forecast anomalies

Subseasonal, seasonal, and decadal ensemble forecasts are typically presented and verified as anomalies with respect to a lead-time-dependent climatological mean to remove the influence of systematic model biases. However, common methods for calculating anomalies result in statistical inconsistencies between forecast and verification anomalies, even for a perfectly reliable ensemble. It is important to account for these systematic effects when evaluating ensemble forecast systems, particularly when tuning a model to improve the variance and/or reliability of forecast anomalies or when comparing spread-error diagnostics between systems with different reforecast periods. Here, we show that unbiased variances and spread/error ratios can be recovered by deriving estimators that are consistent with the values that would be achieved when calculating anomalies relative to the true, but unknown, climatological mean. An elegant alternative is to construct forecast climatologies separately for each member, which ensures that forecast and verification anomalies are defined relative to reference climatological means with the same sampling uncertainty. This alternative approach has no impact on forecast ensemble means but systematically modifies the total variance and ensemble spread of forecast anomalies in such a way that anomaly-based spread/error ratios are unbiased without any explicit correction for climatology sample size. Furthermore, the improved statistical consistency of forecast and verification anomalies means that probabilistic forecast skill is optimized when the underlying forecast is also perfectly reliable. Alternative methods for anomaly calculation can thus impact probabilistic forecast skill, especially when anomalies are defined relative to climatologies with a small sample size. For example, spread/error ratios derived from 5-year reforecasts can vary by +/- 12%$$ \pm 12\% $$, depending on the choice of anomaly calculation method. Finally, we demonstrate the equivalence of anomalies calculated using different methods after applying an unbiased member-by-member statistical calibration.