Improving the performance of GIS polygon overlay computation with MapReduce for spatial big data processing

被引:48
作者
Wang, Yong [1 ]
Liu, Zhenling [1 ]
Liao, Hongyan [1 ]
Li, Chengjun [1 ]
机构
[1] China Univ Geosci, Sch Comp Sci, Wuhan 430074, Peoples R China
来源
CLUSTER COMPUTING-THE JOURNAL OF NETWORKS SOFTWARE TOOLS AND APPLICATIONS | 2015年 / 18卷 / 02期
关键词
GIS; Polygon overlay processing; MapReduce; Grid index; GPC;
D O I
10.1007/s10586-015-0428-x
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
As one of the important operations in Geographic Information System (GIS) spatial analysis, polygon overlay processing is a time-consuming task in many big data cases. In this paper, a specially designed MapReduce algorithm with grid index is proposed to decrease the running time. Our proposed algorithm can reduce the times of calling intersection computation by the aid of grid index. The experiment is carried out on the cloud framework based on Hadoop built by ourselves. Experimental results show that our algorithm with spatial grid index consumes less time than its peer without spatial index. Moreover, the proposed algorithm has an upward speed-up ratio when more nodes of Hadoop framework are used. Nevertheless, with the increase of nodes, the upward trend of speed-up ratio slows down.
引用
收藏
页码:507 / 516
页数:10
相关论文
共 50 条
[21]   A Survey on Geographically Distributed Big-Data Processing Using MapReduce [J].
Dolev, Shlomi ;
Florissi, Patricia ;
Gudes, Ehud ;
Sharma, Shantanu ;
Singer, Ido .
IEEE TRANSACTIONS ON BIG DATA, 2019, 5 (01) :60-80
[22]   An Approach to Enhance the Performance of Hadoop MapReduce Framework for Big Data [J].
Chandra, Subhash ;
Motwani, Deepak .
2016 INTERNATIONAL CONFERENCE ON MICRO-ELECTRONICS AND TELECOMMUNICATION ENGINEERING (ICMETE), 2016, :178-182
[23]   LandQυ2: A MapReduce-Based System for Processing Arable Land Quality Big Data [J].
Yao, Xiaochuang ;
Mokbel, Mohamed E. ;
Ye, Sijing ;
Li, Guoqing ;
Alarabi, Louai ;
Eldawy, Ahmed ;
Zhao, Zuliang ;
Zhao, Long ;
Zhu, Dehai .
ISPRS INTERNATIONAL JOURNAL OF GEO-INFORMATION, 2018, 7 (07)
[24]   The Possibilities of Big GIS Data Processing on the Desktop Computers [J].
Bartonek, Dalibor .
RISE OF BIG SPATIAL DATA, 2017, :273-287
[25]   Big Spatial Data Processing With Apache Spark [J].
Boyi Shangguan ;
Peng Yue ;
Wu, Zhaoyan ;
Jiang, Liangcun .
2017 6TH INTERNATIONAL CONFERENCE ON AGRO-GEOINFORMATICS, 2017, :239-242
[26]   Performance Evaluation of MRDataCube for Data Cube Computation Algorithm Using MapReduce [J].
Lee, Suan ;
Kim, Jinho .
2016 INTERNATIONAL CONFERENCE ON BIG DATA AND SMART COMPUTING (BIGCOMP), 2016, :325-328
[27]   Improving MapReduce Performance by Data Prefetching in Heterogeneous or Shared Environments [J].
Gu, Tao ;
Zuo, Chuang ;
Liao, Qun ;
Yang, Yulu ;
Li, Tao .
INTERNATIONAL JOURNAL OF GRID AND DISTRIBUTED COMPUTING, 2013, 6 (05) :71-81
[28]   Architecture of Efficient Word Processing using Hadoop MapReduce for Big Data Applications [J].
Mandal, Bichitra ;
Sahoo, Ramesh Kumar ;
Sethi, Srinivas .
PROCEEDINGS 2015 INTERNATIONAL CONFERENCE ON MAN AND MACHINE INTERFACING (MAMI), 2015,
[29]   A Performance Analysis of MapReduce Applications on Big Data in Cloud based Hadoop [J].
Gohil, Parth ;
Garg, Dweepna ;
Panchal, Bakul .
2014 INTERNATIONAL CONFERENCE ON INFORMATION COMMUNICATION AND EMBEDDED SYSTEMS (ICICES), 2014,
[30]   Fragmenting Big Data to boost the performance of MapReduce in geographical computing contexts [J].
Cavallo, Marco ;
Di Modica, Giuseppe ;
Polito, Carmelo ;
Tomarchio, Orazio .
2017 3RD INTERNATIONAL CONFERENCE ON BIG DATA INNOVATIONS AND APPLICATIONS (INNOVATE-DATA), 2017, :17-24