The Possibilities of Big GIS Data Processing on the Desktop Computers

被引:0
作者
Bartonek, Dalibor [1 ,2 ]
机构
[1] Brno Univ Technol, Inst Geodesy, Fac Civil Engn, Veveri 330-95, Brno 60200, Czech Republic
[2] European Polytech Inst, Osvobozeni 899, Kunovice 68604, Czech Republic
来源
RISE OF BIG SPATIAL DATA | 2017年
关键词
Big data; GIS; Optimization; Desktop computer;
D O I
10.1007/978-3-319-45123-7_20
中图分类号
P9 [自然地理学]; K9 [地理];
学科分类号
0705 ; 070501 ;
摘要
The paper submits the method how to solve big projects in the sphere of geographic information systems (GIS). Our aim is to answer the question whether we can or cannot solve similar projects on commonly used hardware and software. This method is based on making use of parallelism and optimization of individual processes. The whole GIS project is divided according to the territory principle into the individual projects which can be processed concurrently. In the frame of sub-projects data optimization of main theme is performed. After the finishing of the particular phases of the project a manual check of partial results follows. The final step consists in completing the separate results into common database. The project was solved for the GasNet, Ltd. Company which is a part of a RWE group in the Czech Republic. Input data were datasets of orthophoto with a resolution of 25 cm/pixel, layers of communications of ZABAGED CR and vector sets of the route of line of underground engineering networks. Due to the territorial coverage of the CR with the area of 64,350 km(2), these were massive tasks with total data volume more than 500 GB. The data analysis was carried out in the special created application in Python language with the support of ESRI libraries and also in ArcGIS 10.0 environment.
引用
收藏
页码:273 / 287
页数:15
相关论文
共 50 条
[41]   Prominence of MapReduce in BIG DATA Processing [J].
Pandey, Shweta ;
Tokekar, Vrinda .
2014 FOURTH INTERNATIONAL CONFERENCE ON COMMUNICATION SYSTEMS AND NETWORK TECHNOLOGIES (CSNT), 2014, :555-560
[42]   Computing infrastructure for big data processing [J].
Ling Liu .
Frontiers of Computer Science, 2013, 7 :165-170
[43]   Computing infrastructure for big data processing [J].
Liu, Ling .
FRONTIERS OF COMPUTER SCIENCE, 2013, 7 (02) :165-170
[44]   Big Data Processing on Volunteer Computing [J].
Lv, Zhihan ;
Chen, Dongliang ;
Singh, Amit Kumar .
ACM TRANSACTIONS ON INTERNET TECHNOLOGY, 2021, 21 (04)
[45]   BigGIS: How Big Data Can Shape Next-Generation GIS [J].
Yue, Peng ;
Jiang, Liangcun .
THIRD INTERNATIONAL CONFERENCE ON AGRO-GEOINFORMATICS (AGRO-GEOINFORMATICS 2014), 2014, :413-418
[46]   Research of Big Data Processing Platform [J].
Liu, Xiangju .
GREEN POWER, MATERIALS AND MANUFACTURING TECHNOLOGY AND APPLICATIONS III, PTS 1 AND 2, 2014, 484-485 :922-926
[47]   Diversification on big data in query processing [J].
Meifan Zhang ;
Hongzhi Wang ;
Jianzhong Li ;
Hong Gao .
Frontiers of Computer Science, 2020, 14
[48]   Big data processing framework for manufacturing [J].
Ye, Yinghao ;
Wang, Meilin ;
Yao, Shuhong ;
Jiang, Jarvis N. ;
Liu, Qing .
11TH CIRP CONFERENCE ON INDUSTRIAL PRODUCT-SERVICE SYSTEMS, 2019, 83 :661-664
[49]   The Performance Optimization of Big Data Processing by Adaptive MapReduce Workflow [J].
Li, Wei ;
Tang, Maolin .
IEEE ACCESS, 2022, 10 :79004-79020
[50]   Virtual design of urban planning based on GIS big data and machine learning [J].
Zhu, Bin ;
Zhou, Jie .
JOURNAL OF INTELLIGENT & FUZZY SYSTEMS, 2021, 40 (04) :6263-6273