Backtesting VaR and expectiles with realized scores

Several statistical functionals such as quantiles and expectiles arise naturally as the minimizers of the expected value of a scoring function, a property that is called elicitability (see Gneiting in J Am Stat Assoc 106:746-762,2011 and the references therein). The existence of such scoring functions gives a natural way to compare the accuracy of different forecasting models, and to test comparative hypotheses by means of the Diebold-Mariano test as suggested in a recent work. In this paper we suggest a procedure to test the accuracy of a quantile or expectile forecasting model in an absolute sense, as in the original Basel I backtesting procedure of value-at-risk. To this aim, we study the asymptotic and finite-sample distributions of empirical scores for normal and uniform i.i.d. samples. We compare on simulated data the empirical power of our procedure with alternative procedures based on empirical identification functions (i.e. in the case of VaR the number of violations) and we find an higher power in detecting at least misspecification in the mean. We conclude with a real data example where both backtesting procedures are applied to AR(1)-Garch(1,1) models fitted to SP500 logreturns for VaR and expectiles' forecasts.