Smart-Meter Big Data for Load Forecasting: An Alternative Approach to Clustering

Accurate forecasting of electricity demand is vital to the resilient management of energy systems. Recent efforts in harnessing smart-meter data to improve forecasting accuracy have primarily centered around cluster-based approaches (CBAs), where smart-meter data are grouped into a small number of clusters and separate prediction models are developed for each cluster. The cluster-based predictions are then aggregated to compute the total demand. CBAs have provided promising results compared to conventional approaches that are generally not conducive to integrating smart-meter data. However, CBAs are computationally costly and suffer from the curse of dimensionality, especially under scenarios involving smart-meter data from millions of customers. In this work, we propose an efficient reduced model approach (RMA) that leverages a novel hierarchical dimension reduction algorithm to enable the integration of fine-resolution high-dimensional smart-meter data for millions of customers in load prediction. We demonstrate the applicability of our proposed approach by using data from a utility company, based in Illinois, United States, with more than 3.7 million customers and present model performance in-terms of forecast accuracy. The proposed hierarchical dimension reduction approach enables utilizing the high-resolution data from smart-meters in a scalable manner that is not exploitable otherwise. The results shows significant improvements in forecast accuracy compared to the available approaches that either do not harness fine-resolution data or are not scalable to large-scale smart-meter big data.