Huge Data Analysis and Processing Platform based on Hadoop

SGC puts forward the major initiatives of the construction of strong smart grid and three integrates and five big systems, requiring the informationlization further improves to a level on the basis of the inheritance for fruit and deepening application, having even higher requirements especially for huge information data processing, data search grid. In response to meet the needs of the smart grid to the huge amounts of data processing, this article puts forward building dynamic scalable huge amounts of data processing platform on the basis of Hadoop in the virtualization resources management platform through in-depth analysis and research of the two core key technology, HDFS and MapReduce and gives its technical architecture, implementation plan and case analysis. Compared with traditional Hadoop distributed parallel computing system based on the physical machine deployment, to build Hadoop virtual server templates through virtualization platform not only can quickly complete the Hadoop deployment of distributed parallel computing system and can effectively make use of computing resources.