Real-Time Handling of Network Monitoring Data Using a Data-Intensive Framework

The proper operation and maintenance of a network requires a reliable and efficient monitoring mechanism. The mechanism should handle large amount of monitoring data which are generated by different protocols. In addition, the requirements (e.g. response time, accuracy) imposed by long-term planned queries and short-term ad-hoc queries should be satisfied for multi-tenant computing models. This paper proposes a novel mechanism for scalable storage and real-time processing of monitoring data. This mechanism takes advantage of a data-intensive framework for collecting network flow information records, as well as data points' indexes. The design is not limited to a particular monitoring protocol, since it employs a generic structure for data handling. Thus, it's applicable to a wide variety of monitoring solutions.