An efficient approach for detecting anomalous events in real-time weather datasets

被引:4
作者
Arora, Shruti [1 ]
Rani, Rinkle [1 ]
Saxena, Nitin [1 ]
机构
[1] Thapar Inst Engn & Technol, Dept Comp Sci & Engn, Patiala, Punjab, India
关键词
Apache Kafka; Apache Spark; events; LSTM; RNN; stream processing; NETWORK;
D O I
10.1002/cpe.6707
中图分类号
TP31 [计算机软件];
学科分类号
081202 ; 0835 ;
摘要
Event detection in real-time is applied in diverse domains such as detection of fraudulent activities in commercial transactions, detection of faulty systems in industries, and so forth. Businesses and organizations benefit from the actionable information obtained through various techniques available for anomalous event detection. Real-time event detection is nowadays handled through streaming data frameworks. Traditional approaches effectively handle event detection in real-time but with more false positives, thus, resulting in false alarms. In this article, an efficient approach comprising two components, an offline model and an online event detection pipeline, is proposed to achieve minimum mean absolute error (MAE). An offline module is developed to investigate a variety of deep learning models that prove suitable for event detection in real-time. The experiments conducted with PubNub sensors datasets demonstrate that the long short-term memory unit of recurrent neural networks is the best suitable model for anomalous event detection. The online pipeline module is built using streaming data frameworks to predict the abnormal peaks. It is revealed through the experimental results that the proposed approach efficiently detects anomalous events in real-time and also eliminates false positives.
引用
收藏
页数:15
相关论文
共 66 条
  • [1] Robust real-time unusual event detection using multiple fixed-location monitors
    Adam, Amit
    Rivlin, Ehud
    Shimshoni, Ilan
    Reinitz, David
    [J]. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2008, 30 (03) : 555 - 560
  • [2] Iktishaf: a Big Data Road-Traffic Event Detection Tool Using Twitter and Spark Machine Learning
    Alomari, Ebtesam
    Katib, Iyad
    Mehmood, Rashid
    [J]. MOBILE NETWORKS & APPLICATIONS, 2023, 28 (02) : 603 - 618
  • [3] Machine Learning for Climate Precipitation Prediction Modeling over South America
    Anochi, Juliana Aparecida
    de Almeida, Vinicius Albuquerque
    de Campos Velho, Haroldo Fraga
    [J]. REMOTE SENSING, 2021, 13 (13)
  • [4] [Anonymous], 2010, 2010 IEEE COMPUTER S, DOI [DOI 10.1109/CVPR.2010.5539872, 10.1109/CVPR.2010.5539872]
  • [5] Short-Term Precipitation Forecast Based on the PERSIANN System and LSTM Recurrent Neural NetworksN
    Asanjan, Ata Akbari
    Yang, Tiantian
    Hsu, Kuolin
    Sorooshian, Soroosh
    Lin, Junqiang
    Peng, Qidong
    [J]. JOURNAL OF GEOPHYSICAL RESEARCH-ATMOSPHERES, 2018, 123 (22) : 12543 - 12563
  • [6] Batyuk A, 2016, PROCEEDINGS OF THE 2016 IEEE FIRST INTERNATIONAL CONFERENCE ON DATA STREAM MINING & PROCESSING (DSMP), P345, DOI 10.1109/DSMP.2016.7583573
  • [7] Bhatt RB, 2009, ANNU IEEE IND CONF, P165
  • [8] Birnie C., 2020, ARXIV PREPRINT ARXIV
  • [9] A Proof of Local Convergence for the Adam Optimizer
    Bock, Sebastian
    Weiss, Martin
    [J]. 2019 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS (IJCNN), 2019,
  • [10] SystemML: Declarative Machine Learning on Spark
    Boehm, Matthias
    Dusenberry, Michael W.
    Eriksson, Deron
    Evfimievski, Alexandre V.
    Manshadi, Faraz Makari
    Pansare, Niketan
    Reinwald, Berthold
    Reiss, Frederick R.
    Sen, Prithviraj
    Surve, Arvind C.
    Tatikonda, Shirish
    [J]. PROCEEDINGS OF THE VLDB ENDOWMENT, 2016, 9 (13): : 1425 - 1436