Assessing probabilistic forecasts of volcanic eruption onsets

A method for assessing prospectively the quality of a suite of eruption forecasts is proposed. Any forecast of the next eruption onset from a polygenetic volcano can be converted into a probability distribution for the elapsed time since the forecast is made. This probability distribution, which effectively becomes a statistical P value when the observation is "plugged in," will thus itself have a uniform distribution under the null hypothesis that the forecast correctly describes the process. Given sufficient realizations, which may be on the same or different volcanoes, we can use standard statistical tests, such as the Kolmogorov-Smirnov test, to determine if the forecasts are consistent with the model(s). The use of the Kolmogorov-Smirnov test enables currently open forecasts to be included via the Kaplan-Meier product-limit estimator. While consistent underestimates (or overestimates) of the repose length will result in a median greater (or less) than 0.5, the method also assesses whether the method assigns the correct degree of aleatory variability to the forecast. Note that it is possible for the forecasts to be less precise than claimed. This would be indicated by the median of the sample being around 0.5, but the quartiles being well outside the (0.25, 0.75) interval, for example. The method is illustrated on the author's library of forecasts dating back to 1994, including renewal models and other point processes, on a gallery of approximately 20 volcanoes including Etna, Aso, and Ruapehu.