Streaming Machine Learning for Supporting Data Prefetching in Modern Data Storage Systems

Modern data storage systems optimize data access by distributing data across multiple storage tiers and caches, based on numerous tiering and caching policies. The policies' decisions, and in particular the ones related to data prefetching, can severely impact the performance of the entire storage system. In recent years, various machine learning algorithms have been employed to model access patterns in complex data storage workloads. Even though data storage systems handle a constantly changing stream of file requests, current approaches continue to train their models offline in a batch-based approach. In this paper, we investigate the use of streaming machine learning to support data prefetching decisions in data storage systems as it introduces various advantages such as high training efficiency, high prediction accuracy, and high adaptability to changing workload patterns. After extracting a representative set of features in an online fashion, streaming machine learning models can be trained and tested while the system is running. To validate our methodology, we present one streaming classification model to predict the next file offset to be read in a file. We assess the model's performance using production traces provided by Huawei Technologies and demonstrate that streaming machine learning is a feasible approach with low memory consumption and minimal training delay, facilitating accurate predictions in real-time.