Hadoop real-time monitoring system based on Ganglia, Nagios and MongoDB

Currently, Hadoop is widely used in a growing number of network applications. Based on the advantages of its open source framework for distributed systems, it implements Hadoop distributed file system and Map Re-duce distributed computing framework. Therefore, how to conduct performance monitoring nodes in the cluster to ensure the normal and efficient operation of the whole cluster is one of the key studies on Hadoop. Although there are a large number of third-party monitoring tools to achieve monitoring for Hadoop cluster, limitations of monitoring indicators lead to inadequate monitoring granularity. This article is a supplement of Ganglia, which is a real-time monitoring system. Using JMX interface which Hadoop supports to get more monitoring indicators, we can achieve full performance monitoring of Hadoop cluster when running tasks. Meanwhile, it makes use of the advantages of Nagios and MongoDB, and resolves the problem that Ganglia cannot store data permanently.