Impact of Metadata Server on a Large Scale File System

Big Data, the most popular paradigm, consists of a huge set of data. Big Data storage is built using block-store, file system, object-store and/or hybrid of these systems, cohered with metadata. In this paper, we present role of the metadata server (MDS) in a file system for Big Data. The metadata is large in size for Big Data storage system, and therefore, a standalone metadata server (MDS) cannot hold entire metadata of the storage system. Thus, the metadata servers are augmented to form a clustered metadata server. Moreover, the clustered metadata server (MDS) defines the retrieval process of data from the clustered storage media. Besides, the clustered MDS defines the scalability of Big Data storage. In addition, the clustered MDS offers high efficiency and fast accessing of data in a large scale data storage. The size of Big Data is typically Petabytes to Exabytes. Therefore, there is a strong requirement of efficient and effective clustered MDS to define the access mechanism of Big Data storage. In this paper, we sketch the effect of MDS on very large scale file system.