Root mean square error or mean absolute error? Use their ratio as well

The key statistical properties of the Root Mean Square Error (RMSE) and the Mean Absolute Error (MAE) estimators were derived in this study for zero mean symmetric error distribu-tions. A density function, named the Approximate Root Normal Distribution (ARND), was developed to approximate the distribution of a square root of a normal random variable. This enabled approximating the distribution of the RMSE estimator. The theoretical deriva-tions and the demonstrations on common distributions, a benchmark time series, and a real world data set (with prediction errors generated from ANN and ARIMA models) lead to the following practically useful findings. When comparing errors having the same distri-bution type, RMSE was shown to be preferred for platykurtic distributions, MAE for lep-tokurtic distributions, and either RMSE or MAE for mesokurtic distributions. For different distribution types, however, using the two estimators alone was shown to lead to erro-neous conclusions. The revelation that the estimated RMSE/MSE ratio could identify whether the errors came from platykurtic/mesokurtic/leptokurtic distributions was a use-ful complementary result. Comparison of errors based on, error distributions, sample size and the standard errors of the estimators, was discussed. The proposed procedure for deriving the statistical properties of the two estimators has scope for extension for other distribution types.(c) 2021 Elsevier Inc. All rights reserved.