A Statistical Technique for Online Anomaly Detection for Big Data Streams in Cloud Collaborative Environment

Big data and cloud computing are the two top IT initiatives that are in the mind for industries across the globe. Both innovations keep on evolving. As a delivery model for IT services, cloud computing has the potential to enhance agility and productivity while enabling greater efficiencies and reducing costs. As a result a number of enterprises are building efficient and agile cloud environments, and cloud providers continue to expand service offerings. Many cloud providers offer online collaboration service which is basically loosely-coupled in nature. Online anomaly detection aims to detect anomalies in data flowing in a streaming fashion. Such stream data is commonplace in today's cloud centric collaborations which enables participating domains to dynamically interoperate through sharing and accessing of information. Accordingly to forestall unauthorized disclosure of the shared resources and conceivable misappropriation, there is a need to identify anomalous access requests. To the best of our knowledge, the detection of anomalous access requests in cloud-based collaborations through non-parametric statistical technique has not been studied in earlier works. This paper proposes an online anomaly detection algorithm based on Kolmogorov-Smirnov goodness of fit test to detect anomalous access requests in cloud environment at runtime.