Storage workload modelling by hidden Markov models: Application to Flash memory

A workload analysis technique is presented that processes data from operation type traces and creates a hidden Markov model (HMM) to represent the workload that generated those traces. The HMM can be used to create representative traces for performance models, such as simulators, avoiding the need to repeatedly acquire suitable traces. It can also be used to estimate the transition probabilities and rates of a Markov modulated arrival process directly, for use as input to an analytical performance model of Flash memory. The HMMs obtained from industrial workloads - both synthetic benchmarks, preprocessed by a file translation layer, and real, time-stamped user traces - are validated by comparing their autocorrelation functions and other statistics with those of the corresponding monitored time series. Further, the performance model applications, referred to above, are illustrated by numerical examples. (C) 2011 Elsevier B.V. All rights reserved.