The diagonal score: Definition, properties, and interpretations

The diagonal score is proposed as a new scoring rule for the assessment of univariate probabilistic forecasts. First, a general framework for the definition of proper user-oriented scores is set up using the concept of elementary scoring rules. Building on this fundamental notion, a complete overview of forecast skill is provided by a new verification tool. The forecast skill card introduced displays the forecast value as a function of two decision parameters, a probability level and an event threshold, simultaneously. The diagonal score emerges as a summary score focusing on the diagonal of the skill card, that is, by fixing the relationship between the event base rate and forecast probability level. The properties of the new score, as well as interpretations from the perspectives of users and developers, are discussed based on synthetic datasets and 2 m temperature forecasts of the operational ECMWF ensemble prediction system.