A visual big data system for the prediction of weather-related variables: Jordan-Spain case study

被引:2
作者
Aljawarneh, Shadi [1 ]
Lara, Juan A. [2 ]
Yassein, Muneer Bani [1 ]
机构
[1] Jordan Univ Sci & Technol, Fac Comp & Informat Technol, JUST, POB 3030, Irbid 22110, Jordan
[2] Madrid Open Univ, UDIMA, Sch Comp Sci, KM 38,500,Via Serv 15, Madrid 28400, Spain
关键词
Big data; Weather forecasting; Data mining; Information fusion; MongoDB;
D O I
10.1007/s11042-020-09848-9
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
The Meteorology is a field where huge amounts of data are generated, mainly collected by sensors at weather stations, where different variables can be measured. Those data have some particularities such as high volume and dimensionality, the frequent existence of missing values in some stations, and the high correlation between collected variables. In this regard, it is crucial to make use of Big Data and Data Mining techniques to deal with those data and extract useful knowledge from them that can be used, for instance, to predict weather phenomena. In this paper, we propose a visual big data system that is designed to deal with high amounts of weather-related data and lets the user analyze those data to perform predictive tasks over the considered variables (temperature and rainfall). The proposed system collects open data and loads them onto a local NoSQL database fusing them at different levels of temporal and spatial aggregation in order to perform a predictive analysis using univariate and multivariate approaches as well as forecasting based on training data from neighbor stations in cases with high rates of missing values. The system has been assessed in terms of usability and predictive performance, obtaining an overall normalized mean squared error value of 0.00013, and an overall directional symmetry value of nearly 0.84. Our system has been rated positively by a group of experts in the area (all aspects of the system except graphic desing were rated 3 or above in a 1-5 scale). The promising preliminary results obtained demonstrate the validity of our system and invite us to keep working on this area.
引用
收藏
页码:13103 / 13139
页数:37
相关论文
共 47 条
  • [1] A Big Data Prediction Framework for Weather Forecast Using MapReduce Algorithm
    Adam, Khalid
    Majid, Mazlina Abdul
    Fakherldin, Mohammed Adam Ibrahim
    Zain, Jasni Mohamed
    [J]. ADVANCED SCIENCE LETTERS, 2017, 23 (11) : 11138 - 11143
  • [2] Aggarwal C, 2014, DATA CLASSIFICATIONA
  • [3] The adequacy of stochastically generated climate time series for water resources systems risk and performance assessment
    Alodah, Abdullah
    Seidou, Ousmane
    [J]. STOCHASTIC ENVIRONMENTAL RESEARCH AND RISK ASSESSMENT, 2019, 33 (01) : 253 - 269
  • [4] Ambigavathi M, 2020, INTELLIGENT COMMUNIC, V989
  • [5] Ordinal Regression Methods: Survey and Experimental Study
    Antonio Gutierrez, Pedro
    Perez-Ortiz, Maria
    Sanchez-Monedero, Javier
    Fernandez-Navarro, Francisco
    Hervas-Martinez, Cesar
    [J]. IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, 2016, 28 (01) : 127 - 146
  • [7] Benchmarking big data systems: A survey
    Bajaber, Fuad
    Sakr, Sherif
    Batarfi, Omar
    Altalhi, Abdulrahman
    Barnawi, Ahmed
    [J]. COMPUTER COMMUNICATIONS, 2020, 149 : 241 - 251
  • [8] Booz J, 2019, INT CONF COMPUT NETW, P697, DOI [10.1109/ICCNC.2019.8685584, 10.1109/iccnc.2019.8685584]
  • [9] Random forests
    Breiman, L
    [J]. MACHINE LEARNING, 2001, 45 (01) : 5 - 32
  • [10] Chodorow Kristina., 2010, MONGODB DEFINITIVE G, V1st