Machine learning and deep learning prediction models for time-series: a comparative analytical study for the use case of the UK short-term electricity price prediction

Electricity price prediction has an imperative role in the UK energy market among energy trading organisations. The price prediction directly impacts organisational policy for profitable electricity trading, better bidding plans, and the optimisation of energy storage devices for any surplus energy. Business organisations always look for the use of price-prediction models with higher accuracy to help them maximise benefits. With the enhancement of Internet of Things (IoT) technology, data availability on energy demand, and hence the associated price prediction modelling has become more effective tools than before. However, price prediction has been a challenging task because of the uncertainty in the demand and supply and other external factors such as weather, and gas prices as these factors can influence the fluctuation of electricity prices. In this regard, the selection of an appropriate prediction model is crucial for business organisations. In this paper, an analytical study has been presented to predict short-term electricity prices in the UK market as a use case for a UK-based energy trading company. ARIMA, Prophet, XGBoost as well as Convolution Neural Networks (CNN), Recurrent Neural Networks (RNN), and Long-Short Term Memory (LSTM) algorithms have been analysed. In this study, UK Market Index Data (MID) from Elexon API data has been used that represent half-hourly electricity prices. In addition, gas prices, and initial demand out-turn data, as external factors, are introduced into the models for improving the accuracy and performance of these models. The comparative analysis shows that the ARIMA can handle only one external factor in its prediction model, while the Prophet and XGBoost can incorporate multiple external regressors in their models. Also, the models based on deep learning algorithms can deal with univariate and multivariate time series. The comparative analysis also revealed that the XGBoost model has better performance than the ARIMA and Prophet models, as has been found in earlier studies. With the extended analysis, it has been found that deep learning models outperform ARIMA, Prophet, and XGBoost models in terms of prediction accuracy. This extended comparative analysis gives the flexibility to choose the appropriate model selection for any organisation working in analogous business scenarios as of the business use case of this study. © The Author(s) 2024.