Anomaly Detection with Machine Learning Algorithms and Big Data in Electricity Consumption

被引:38
|
作者
Oprea, Simona-Vasilica [1 ]
Bara, Adela [1 ]
Puican, Florina Camelia [1 ]
Radu, Ioan Cosmin [2 ]
机构
[1] Bucharest Univ Econ Studies, Dept Econ Informat & Cybernet, Romana Sq 6, Bucharest 010374, Romania
[2] Univ Politehn Bucuresti, Dept Engn Foreign Languages, Splaiul Independent 313, Bucharest 060042, Romania
关键词
anomaly detection; unsupervised and supervised machine learning; big data; smart grid; fraud detection; DETECTION FRAMEWORK; THEFT DETECTION; FRAUD DETECTION; ENERGY THEFT;
D O I
10.3390/su131910963
中图分类号
X [环境科学、安全科学];
学科分类号
08 ; 0830 ;
摘要
When analyzing smart metering data, both reading errors and frauds can be identified. The purpose of this analysis is to alert the utility companies to suspicious consumption behavior that could be further investigated with on-site inspections or other methods. The use of Machine Learning (ML) algorithms to analyze consumption readings can lead to the identification of malfunctions, cyberattacks interrupting measurements, or physical tampering with smart meters. Fraud detection is one of the classical anomaly detection examples, as it is not easy to label consumption or transactional data. Furthermore, frauds differ in nature, and learning is not always possible. In this paper, we analyze large datasets of readings provided by smart meters installed in a trial study in Ireland by applying a hybrid approach. More precisely, we propose an unsupervised ML technique to detect anomalous values in the time series, establish a threshold for the percentage of anomalous readings from the total readings, and then label that time series as suspicious or not. Initially, we propose two types of algorithms for anomaly detection for unlabeled data: Spectral Residual-Convolutional Neural Network (SR-CNN) and an anomaly trained model based on martingales for determining variations in time-series data streams. Then, the Two-Class Boosted Decision Tree and Fisher Linear Discriminant analysis are applied on the previously processed dataset. By training the model, we obtain the required capabilities of detecting suspicious consumers proved by an accuracy of 90%, precision score of 0.875, and F1 score of 0.894.
引用
收藏
页数:20
相关论文
共 50 条
  • [31] A survey of big data architectures and machine learning algorithms in healthcare
    Manogaran G.
    Lopez D.
    International Journal of Biomedical Engineering and Technology, 2017, 25 (2-4) : 182 - 211
  • [32] A Review at Machine Learning Algorithms Targeting Big Data Challenges
    Rathor, Abhinav
    Gyanchandani, Manasi
    2017 INTERNATIONAL CONFERENCE ON ELECTRICAL, ELECTRONICS, COMMUNICATION, COMPUTER, AND OPTIMIZATION TECHNIQUES (ICEECCOT), 2017, : 753 - 758
  • [33] Comparison of machine learning algorithms for classification of Big Data sets
    Singh, Barkha
    Indu, Sreedevi
    Majumdar, Sudipta
    THEORETICAL COMPUTER SCIENCE, 2025, 1024
  • [34] Machine Learning with Big Data An Efficient Electricity Generation Forecasting System
    Rahman, Mohammad Naimur
    Esmailpour, Amir
    Zhao, Junhui
    BIG DATA RESEARCH, 2016, 5 : 9 - 15
  • [35] An anomaly Detection Method for Electricity Consumption Data Based on CNN-BiLSTMAttention
    Sun, Jinkai
    Ren, Yulu
    Zhang, Junwei
    Chen, Xiaofang
    JOURNAL OF ELECTRICAL SYSTEMS, 2024, 20 (02) : 1924 - 1932
  • [36] Variational autoencoders for anomaly detection in the behaviour of the elderly using electricity consumption data
    Gonzalez, Daniel
    Patricio, Miguel A.
    Berlanga, Antonio
    Molina, Jose M.
    EXPERT SYSTEMS, 2022, 39 (04)
  • [37] Anomaly detection using unsupervised machine learning algorithms: A simulation study
    Agyemang, Edmund Fosu
    SCIENTIFIC AFRICAN, 2024, 26
  • [38] Corporate network anomaly detection methodology utilizing machine learning algorithms
    Baisholan, Nazerke
    Baisholanova, Karlygash
    Kubayev, Kazila
    Alimzhanova, Zhanna
    Baimuldina, Nazira
    SMART SCIENCE, 2024, 12 (04) : 666 - 678
  • [39] Anomaly detection for atomic clocks using unsupervised machine learning algorithms
    Chen, Edwin
    Charbonneau, Andre
    Gertsvolf, Marina
    Wang, Yunli
    METROLOGIA, 2024, 61 (05)
  • [40] Feature Reduction and Anomaly Detection in IoT Using Machine Learning Algorithms
    Hamdan, Adel
    Tahboush, Muhannad
    Adawy, Mohammad
    Alwada'n, Tariq
    Ghwanmeh, Sameh
    INTERNATIONAL JOURNAL OF ADVANCED COMPUTER SCIENCE AND APPLICATIONS, 2025, 16 (01) : 463 - 470