A Practical Yet Accurate Real-Time Statistical Analysis Library for Hydrologic Time-Series Big Data

被引:2
作者
Sun, Jun [1 ]
Ye, Feng [1 ]
Nedjah, Nadia [2 ]
Zhang, Ming [3 ]
Xu, Dong [4 ]
机构
[1] Hohai Univ, Sch Comp & Informat, Nanjing 211100, Peoples R China
[2] Univ Estado Rio De Janeiro, Engn Fac, Dept Elect Engn & Telecommun, Rua Sao Francisco Xavier 524, BR-20550013 Rio De Janeiro, Brazil
[3] Water Resources Dept Jiangsu Prov, Nanjing 210029, Peoples R China
[4] Hohai Univ, Coll Water Conservancy & Hydropower Engn, Nanjing 211100, Peoples R China
基金
中国国家自然科学基金;
关键词
hydrologic information; statistical analysis; Flink; stream data; time series;
D O I
10.3390/w15040708
中图分类号
X [环境科学、安全科学];
学科分类号
08 ; 0830 ;
摘要
Using different statistical analysis methods to examine hydrologic time-series data is the basis of accurate hydrologic status analysis. With the wide application of the Internet of Things and sensor technologies, traditional statistical analysis methods are unable to meet the demand for real-time and accurate hydrologic data analysis. The existing mainstream big-data analysis platforms lack analysis methods oriented to hydrologic data. In this context, a real-time statistical analysis library based on the new generation of big data processing engine Flink, called HydroStreamingLib, was proposed and implemented. Furthermore, in order to prove the efficiency and handiness of the proposed library, a real-time statistical analysis system of hydrologic stream data was developed based on the concepts available in the proposed library. The results showed that HydroStreamingLib provides users with an efficient, real-time statistical verification method, thus extending the application capabilities of Flink Ecology in some specific fields.
引用
收藏
页数:17
相关论文
共 29 条
[1]  
Abdi H., 2010, ENCY RES DESIGN, V2, P897, DOI DOI 10.4135/9781412961288
[2]  
Alcalde-Barros A., 2019, Big Data Anal., V4, DOI [10.1186/s41044-019-0041-8, DOI 10.1186/S41044-019-0041-8]
[3]  
AZIZ K, 2018, 2018 4 INT C OPT APP, P1, DOI DOI 10.1109/ICOA.2018.8370593
[4]   A toolbox for visualizing trends in large-scale environmental data [J].
Bromssen, Claudia von ;
Betner, Staffan ;
Folster, Jens ;
Eklof, Karin .
ENVIRONMENTAL MODELLING & SOFTWARE, 2021, 136 (136)
[5]  
Carbone P, 2015, Data Engineering, V38, DOI DOI 10.1109/IC2EW.2016.56
[6]  
Chalapathy R., 2019, ARXIV
[7]   Statistical normality and homogeneity of a 71-year rainfall dataset for the state of Rio de Janeiro-Brazil [J].
de Gois, Givanildo ;
de Oliveira-Junior, Jose Francisco ;
da Silva Junior, Carlos Antonio ;
Sobral, Bruno Serafini ;
de Bodas Terassi, Paulo Miguel ;
Junior, Antonio Herbete Sousa Leonel .
THEORETICAL AND APPLIED CLIMATOLOGY, 2020, 141 (3-4) :1573-1591
[8]  
Goldstein M., 2012, Proceedings of the 35th German Conference on Artificial Intelligence (KI'12), P59
[9]  
Jinyi Chen, 2021, Signal and Information Processing, Networking and Computers. Proceedings of the 7th International Conference on Signal and Information Processing, Networking and Computers (ICSINC). Lecture Notes in Electrical Engineering (LNEE 677), P827, DOI 10.1007/978-981-33-4102-9_99
[10]   Scalable Analytics on Fast Data [J].
Kipf, Andreas ;
Pandey, Varun ;
Boettcher, Jan ;
Braun, Lucas ;
Neumann, Thomas ;
Kemper, Alfons .
ACM TRANSACTIONS ON DATABASE SYSTEMS, 2019, 44 (01)