Fast clustering and anomaly detection technique for large-scale power data stream

被引：0

作者：

Wang G. ^{[1
]}

Zhou G. ^{[2
]}

Zhao H. ^{[1
]}

Mi Z. ^{[1
]}

机构：

[1] School of Electrical & Electronic Engineering, North China Electric Power University, Baoding

[2] Skill Training Center, State Grid Jibei Electric Power Company Limited, Baoding

来源：

Dianli Xitong Zidonghua/Automation of Electric Power Systems | 2016年 / 40卷 / 24期

基金：

中国国家自然科学基金;

关键词：

Anomaly detection; Cluster; Electricity consumption behavior; Stream computing; Streaming data;

D O I：

10.7500/AEPS20160123002

中图分类号：

学科分类号：

摘要：

With the large-scale data stream recently emerging in power systems, utilizing stream computing technology to improve power system real-time and safety has become a critical requirement. For the large-scale data stream of power consumption information collection system, fast clustering technology and anomaly detection technology for streaming data is studied. With reference to the distributed stream computing platform Spark Streaming, Streaming DBSCAN algorithm is designed and implemented by taking advantage of longitudinal time and transverse space clustering features exhibited in the electricity consumption behavior, which means that the same cluster of users have similar power consuming pattern, and one user has similar historical power consuming data. The streaming DBSCAN algorithm is able to achieve fast anomaly detection of a large-scale power data stream. The experimental environment in support of large-scale data stream processing is set up, which can support and validate the effectiveness of the algorithm. © 2016 Automation of Electric Power Systems Press.

引用

页码：27 / 33

页数：6

共 15 条

[1]

Wang J., Ji Z., Shi M., Et al., Scenario analysis and application research on big data in smart power distribution and consumption systems, Proceedings of the CSEE, 35, 8, pp. 1829-1836, (2015)

[2]

Zhao L., Hou X., Hu J., Et al., Improved k-means algorithm based analysis on massive data of intelligent power utilization, Power System Technology, 38, 10, pp. 2715-2720, (2014)

[3]

Song Y., Zhou G., Zhu Y., Et al., Storage optimization and parallel processing of condition monitoring big data of transmission and transforming equipment based on cloud platform, Proceedings of the CSEE, 35, 2, pp. 255-267, (2015)

[4]

Hu J., Zhu E., Du X., Et al., Application status and development trend of power consumption information collection system, Automation of Electric Power Systems, 38, 2, pp. 131-135, (2014)

[5]

Sun D., Zhang G., Zheng W., Big data stream computing: technologies and instances, Journal of Software, 4, pp. 839-862, (2014)

[6]

Zhao J., Yang G., Mu L., Et al., Research on the application of the data stream technology in grid automation, Power System Technology, 35, 8, pp. 6-11, (2011)

[7]

He Y., Luo Y., Tu G., Design of the frame of data stream management system for the energy management system, Automation of Electric Power Systems, 30, 24, pp. 33-38, (2006)

[8]

Lim L., Misra A., Mo T.L., Adaptive data acquisition strategies for energy-efficient, smartphone-based, continuous processing of sensor streams, Distributed and Parallel Databases, 31, 2, pp. 321-351, (2013)

[9]

Zaharia M., Chowdhury M., Das T., Resilient distributed datasets: a fault-tolerant abstraction for in-memory cluster computing, The 9th USENIX Symposium on Networked Systems Design and Implementation (NSDI), pp. 15-28

[10]

De Amorim R.C., Mirkin B., Minkowski M., Feature weighting and anomalous cluster initializing in k-means clustering, Pattern Recognition, 45, 3, pp. 1061-1075, (2012)

← 1 2 →