A Framework for Benchmarking Machine Learning Methods Using Linear Models for Univariate Time Series Prediction

Time series prediction has been attracting interest of researchers due to its increasing importance in decision-making activities in many fields of knowledge. The demand for better accuracy in time series prediction furthered the arising of many machine learning time series prediction methods (MLM). Choosing a suitable method for a particular dataset is a challenge and demands established benchmark methods (BM) for performance assessment. Suppose a particular BM is selected, and an experimental comparison is made with a particular MLM. If the latter does not provide better prediction results for the same dataset, this indicates that some improvements are needed for the MLM. Regarding this matter, adopting a well-established, easy to interpret, and tuned BM is desirable. This paper presents a framework for systematic benchmarking some MLM against well-known Linear Methods (LM), namely Polynomial Regression and models in the ARIMA family, used as BM for univariate time series prediction. We implemented such a framework within the R-Package named TSPred. This implementation was evaluated using a wide number of datasets from past prediction competitions. The results show that fittest LM provided by TSPred are adequate BM for univariate time series predictions.