When local becomes global: An application study of data consistency in a networked world

As users and companies depend increasingly on shared, networked information services, and as companies and customers become more international, computer systems will need to keep pace with a global scale. We believe that we will continue to see growth in large data centers and service providers as new information services arise and existing services are consolidated on one hand (for ease of management, outsourcing, and reduced duplication), and further distributed on the other hand (for fault-tolerance of critical services and to accommodate the global reach of companies and customers). This paper looks at storage systems in such a networked global environment, studies the characteristics of applications' demands on the storage system and quantifies how these demands might be supported in such an environment. Our study focusses on a collection of traces from several large commercial workloads taken on single, centralized servers, and explores the inherent consistency and performance requirements if these same applications were moved into a networked distributed setting. Rather than focussing on applications or services "designed for the Web", we look at applications designed to operate on local storage and examine how well or poorly they would fare if they were migrated to a distributed setting. Our results indicate that in many cases, the "inherent" level of consistency required and the "true" amount of sharing is sufficiently low that distribution may be feasible with reasonable performance, and without wholesale application changes.