An efficient approach for detecting anomalous events in real-time weather datasets

被引:4
作者
Arora, Shruti [1 ]
Rani, Rinkle [1 ]
Saxena, Nitin [1 ]
机构
[1] Thapar Inst Engn & Technol, Dept Comp Sci & Engn, Patiala, Punjab, India
关键词
Apache Kafka; Apache Spark; events; LSTM; RNN; stream processing; NETWORK;
D O I
10.1002/cpe.6707
中图分类号
TP31 [计算机软件];
学科分类号
081202 ; 0835 ;
摘要
Event detection in real-time is applied in diverse domains such as detection of fraudulent activities in commercial transactions, detection of faulty systems in industries, and so forth. Businesses and organizations benefit from the actionable information obtained through various techniques available for anomalous event detection. Real-time event detection is nowadays handled through streaming data frameworks. Traditional approaches effectively handle event detection in real-time but with more false positives, thus, resulting in false alarms. In this article, an efficient approach comprising two components, an offline model and an online event detection pipeline, is proposed to achieve minimum mean absolute error (MAE). An offline module is developed to investigate a variety of deep learning models that prove suitable for event detection in real-time. The experiments conducted with PubNub sensors datasets demonstrate that the long short-term memory unit of recurrent neural networks is the best suitable model for anomalous event detection. The online pipeline module is built using streaming data frameworks to predict the abnormal peaks. It is revealed through the experimental results that the proposed approach efficiently detects anomalous events in real-time and also eliminates false positives.
引用
收藏
页数:15
相关论文
共 66 条
[31]  
Le Noac'h P, 2017, IEEE INT CONF BIG DA, P4803, DOI 10.1109/BigData.2017.8258548
[32]   Towards a multi-layers anomaly detection framework for analyzing network traffic [J].
Li, Bo ;
Zhang, Simin ;
Li, Ke .
CONCURRENCY AND COMPUTATION-PRACTICE & EXPERIENCE, 2017, 29 (14)
[33]  
Li XH, 2017, PROCEEDINGS OF THE 26TH INTERNATIONAL CONFERENCE ON WORLD WIDE WEB (WWW'17), P685
[34]   Real-Time Cuffless Continuous Blood Pressure Estimation Using Deep Learning Model [J].
Li, Yung-Hui ;
Harfiya, Latifa Nabila ;
Purwandari, Kartika ;
Lin, Yue-Der .
SENSORS, 2020, 20 (19) :1-19
[35]  
Liu Y., 2016, Application of Deep Convolutional Neural Networks for Detecting Extreme Weather in Climate Datasets
[36]   Abnormal Event Detection at 150 FPS in MATLAB [J].
Lu, Cewu ;
Shi, Jianping ;
Jia, Jiaya .
2013 IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2013, :2720-2727
[37]  
Malhotra P., 2016, ACM SIGKDD WORKSH MA
[38]   Twentieth century global glacier mass change: an ensemble-based model reconstruction [J].
Malles, Jan-Hendrik ;
Marzeion, Ben .
CRYOSPHERE, 2021, 15 (07) :3135-3157
[39]   Predicting global patterns of long-term climate change from short-term simulations using machine learning [J].
Mansfield, L. A. ;
Nowack, P. J. ;
Kasoar, M. ;
Everitt, R. G. ;
Collins, W. J. ;
Voulgarakis, A. .
NPJ CLIMATE AND ATMOSPHERIC SCIENCE, 2020, 3 (01)
[40]  
Marchi E, 2015, IEEE IJCNN