Anomaly Detection with Machine Learning Algorithms and Big Data in Electricity Consumption

被引:38
|
作者
Oprea, Simona-Vasilica [1 ]
Bara, Adela [1 ]
Puican, Florina Camelia [1 ]
Radu, Ioan Cosmin [2 ]
机构
[1] Bucharest Univ Econ Studies, Dept Econ Informat & Cybernet, Romana Sq 6, Bucharest 010374, Romania
[2] Univ Politehn Bucuresti, Dept Engn Foreign Languages, Splaiul Independent 313, Bucharest 060042, Romania
关键词
anomaly detection; unsupervised and supervised machine learning; big data; smart grid; fraud detection; DETECTION FRAMEWORK; THEFT DETECTION; FRAUD DETECTION; ENERGY THEFT;
D O I
10.3390/su131910963
中图分类号
X [环境科学、安全科学];
学科分类号
08 ; 0830 ;
摘要
When analyzing smart metering data, both reading errors and frauds can be identified. The purpose of this analysis is to alert the utility companies to suspicious consumption behavior that could be further investigated with on-site inspections or other methods. The use of Machine Learning (ML) algorithms to analyze consumption readings can lead to the identification of malfunctions, cyberattacks interrupting measurements, or physical tampering with smart meters. Fraud detection is one of the classical anomaly detection examples, as it is not easy to label consumption or transactional data. Furthermore, frauds differ in nature, and learning is not always possible. In this paper, we analyze large datasets of readings provided by smart meters installed in a trial study in Ireland by applying a hybrid approach. More precisely, we propose an unsupervised ML technique to detect anomalous values in the time series, establish a threshold for the percentage of anomalous readings from the total readings, and then label that time series as suspicious or not. Initially, we propose two types of algorithms for anomaly detection for unlabeled data: Spectral Residual-Convolutional Neural Network (SR-CNN) and an anomaly trained model based on martingales for determining variations in time-series data streams. Then, the Two-Class Boosted Decision Tree and Fisher Linear Discriminant analysis are applied on the previously processed dataset. By training the model, we obtain the required capabilities of detecting suspicious consumers proved by an accuracy of 90%, precision score of 0.875, and F1 score of 0.894.
引用
收藏
页数:20
相关论文
共 50 条
  • [21] Analysis of Machine Learning Algorithms for Anomaly Detection on Edge Devices
    Huc, Aleks
    Salej, Jakob
    Trebar, Mira
    SENSORS, 2021, 21 (14)
  • [22] Machine Learning-Driven Algorithms for Network Anomaly Detection
    Islam, Md Sirajul
    Rouf, Mohammad Abdur
    Parvez, A. H. M. Shahariar
    Podder, Prajoy
    INVENTIVE COMPUTATION AND INFORMATION TECHNOLOGIES, ICICIT 2021, 2022, 336 : 493 - 507
  • [23] Contrastive learning for efficient anomaly detection in electricity load data
    Choubey, Mohit
    Chaurasiya, Rahul Kumar
    Yadav, J. S.
    SUSTAINABLE ENERGY GRIDS & NETWORKS, 2025, 42
  • [24] A high-throughput architecture for anomaly detection in streaming data using machine learning algorithms
    Surianarayanan C.
    Kunasekaran S.
    Chelliah P.R.
    International Journal of Information Technology, 2024, 16 (1) : 493 - 506
  • [25] Robust Anomaly Detection Algorithms for Real-time Big Data Comparison of algorithms
    Hasani, Zirije
    2017 6TH MEDITERRANEAN CONFERENCE ON EMBEDDED COMPUTING (MECO), 2017, : 449 - 454
  • [26] Testing of algorithms for anomaly detection in Big data using apache spark
    Lighari, Sheeraz Niaz
    Hussain, Dil Muhammad Akbar
    2017 9TH INTERNATIONAL CONFERENCE ON COMPUTATIONAL INTELLIGENCE AND COMMUNICATION NETWORKS (CICN), 2017, : 97 - 100
  • [27] A Comparative Analysis of Unbalanced Data Handling Techniques for Machine Learning Algorithms to Electricity Theft Detection
    Pereira, Jeanne
    Saraiva, Filipe
    2020 IEEE CONGRESS ON EVOLUTIONARY COMPUTATION (CEC), 2020,
  • [28] Big Data Platform for Smart Grids Power Consumption Anomaly Detection
    Lipcak, Peter
    Macak, Martin
    Rossi, Bruno
    PROCEEDINGS OF THE 2019 FEDERATED CONFERENCE ON COMPUTER SCIENCE AND INFORMATION SYSTEMS (FEDCSIS), 2019, : 771 - 780
  • [29] Anomaly Detection Technique for Intrusion Detection in SDN Environment using Continuous Data Stream Machine Learning Algorithms
    Lima Ribeiro, Admilson de Ribamar
    Carvalho Santos, Reneilson Yves
    Alves Nascimento, Anderson Clayton
    2021 15TH ANNUAL IEEE INTERNATIONAL SYSTEMS CONFERENCE (SYSCON 2021), 2021,
  • [30] Machine Learning Algorithms for Big Data Applications With Policy Implementation
    Wu, Jianzu
    Zhang, Kunxin
    JOURNAL OF ORGANIZATIONAL AND END USER COMPUTING, 2022, 34 (03)