Improving the performance of GIS polygon overlay computation with MapReduce for spatial big data processing

被引:48
作者
Wang, Yong [1 ]
Liu, Zhenling [1 ]
Liao, Hongyan [1 ]
Li, Chengjun [1 ]
机构
[1] China Univ Geosci, Sch Comp Sci, Wuhan 430074, Peoples R China
来源
CLUSTER COMPUTING-THE JOURNAL OF NETWORKS SOFTWARE TOOLS AND APPLICATIONS | 2015年 / 18卷 / 02期
关键词
GIS; Polygon overlay processing; MapReduce; Grid index; GPC;
D O I
10.1007/s10586-015-0428-x
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
As one of the important operations in Geographic Information System (GIS) spatial analysis, polygon overlay processing is a time-consuming task in many big data cases. In this paper, a specially designed MapReduce algorithm with grid index is proposed to decrease the running time. Our proposed algorithm can reduce the times of calling intersection computation by the aid of grid index. The experiment is carried out on the cloud framework based on Hadoop built by ourselves. Experimental results show that our algorithm with spatial grid index consumes less time than its peer without spatial index. Moreover, the proposed algorithm has an upward speed-up ratio when more nodes of Hadoop framework are used. Nevertheless, with the increase of nodes, the upward trend of speed-up ratio slows down.
引用
收藏
页码:507 / 516
页数:10
相关论文
共 50 条
[31]   Fragmenting Big Data to boost the performance of MapReduce in geographical computing contexts [J].
Cavallo, Marco ;
Di Modica, Giuseppe ;
Polito, Carmelo ;
Tomarchio, Orazio .
2017 3RD INTERNATIONAL CONFERENCE ON BIG DATA INNOVATIONS AND APPLICATIONS (INNOVATE-DATA), 2017, :17-24
[32]   AMPO: Algorithm for MapReduce Performance Optimization for Enhancing Big Data Analytics [J].
Yambem, Nandita ;
Nandakumar, A. N. .
2017 INTERNATIONAL CONFERENCE ON ELECTRICAL, ELECTRONICS, COMMUNICATION, COMPUTER, AND OPTIMIZATION TECHNIQUES (ICEECCOT), 2017, :717-723
[33]   Big data pre-processing methods with vehicle driving data using MapReduce techniques [J].
Wonhee Cho ;
Eunmi Choi .
The Journal of Supercomputing, 2017, 73 :3179-3195
[34]   Big data pre-processing methods with vehicle driving data using MapReduce techniques [J].
Cho, Wonhee ;
Choi, Eunmi .
JOURNAL OF SUPERCOMPUTING, 2017, 73 (07) :3179-3195
[35]   Effective Pre-processing Methods with DTG Big Data by Using MapReduce Techniques [J].
Cho, Wonhee ;
Choi, Eunmi .
ADVANCES IN COMPUTER SCIENCE AND UBIQUITOUS COMPUTING, 2017, 421 :389-395
[36]   A NEW ENCRYPTION SCHEME FOR PERFORMANCE IMPROVEMENT IN BIG DATA ENVIRONMENT USING MAPREDUCE [J].
Algaradi, Thoyazan Sultan ;
Rama, B. .
JOURNAL OF ENGINEERING SCIENCE AND TECHNOLOGY, 2021, 16 (05) :3772-3791
[37]   Improving Node-Level MapReduce Performance Using Processing-in-Memory Technologies [J].
Islam, Mahzabeen ;
Scrbak, Marko ;
Kavi, Krishna M. ;
Ignatowski, Mike ;
Jayasena, Nuwan .
EURO-PAR 2014: PARALLEL PROCESSING WORKSHOPS, PT II, 2014, 8806 :425-437
[38]   Interactive Analytical Processing in Big Data Systems: A Cross-Industry Study of MapReduce Workloads [J].
Chen, Yanpei ;
Alspaugh, Sara ;
Katz, Randy .
PROCEEDINGS OF THE VLDB ENDOWMENT, 2012, 5 (12) :1802-1813
[39]   RETRACTED ARTICLE: Detecting straggler MapReduce tasks in big data processing infrastructure by neural network [J].
Amir Javadpour ;
Guojun Wang ;
Samira Rezaei ;
Kuan-Ching Li .
The Journal of Supercomputing, 2020, 76 :6969-6993
[40]   Scaling up MapReduce-based Big Data Processing on Multi-GPU systems [J].
Hai Jiang ;
Yi Chen ;
Zhi Qiao ;
Tien-Hsiung Weng ;
Kuan-Ching Li .
Cluster Computing, 2015, 18 :369-383