The Possibilities of Big GIS Data Processing on the Desktop Computers

被引:0
作者
Bartonek, Dalibor [1 ,2 ]
机构
[1] Brno Univ Technol, Inst Geodesy, Fac Civil Engn, Veveri 330-95, Brno 60200, Czech Republic
[2] European Polytech Inst, Osvobozeni 899, Kunovice 68604, Czech Republic
来源
RISE OF BIG SPATIAL DATA | 2017年
关键词
Big data; GIS; Optimization; Desktop computer;
D O I
10.1007/978-3-319-45123-7_20
中图分类号
P9 [自然地理学]; K9 [地理];
学科分类号
0705 ; 070501 ;
摘要
The paper submits the method how to solve big projects in the sphere of geographic information systems (GIS). Our aim is to answer the question whether we can or cannot solve similar projects on commonly used hardware and software. This method is based on making use of parallelism and optimization of individual processes. The whole GIS project is divided according to the territory principle into the individual projects which can be processed concurrently. In the frame of sub-projects data optimization of main theme is performed. After the finishing of the particular phases of the project a manual check of partial results follows. The final step consists in completing the separate results into common database. The project was solved for the GasNet, Ltd. Company which is a part of a RWE group in the Czech Republic. Input data were datasets of orthophoto with a resolution of 25 cm/pixel, layers of communications of ZABAGED CR and vector sets of the route of line of underground engineering networks. Due to the territorial coverage of the CR with the area of 64,350 km(2), these were massive tasks with total data volume more than 500 GB. The data analysis was carried out in the special created application in Python language with the support of ESRI libraries and also in ArcGIS 10.0 environment.
引用
收藏
页码:273 / 287
页数:15
相关论文
共 50 条
[21]   Taxonomy for Humans or Computers? Cognitive Pragmatics for Big Data [J].
Sterner B. ;
Franz N.M. .
Biological Theory, 2017, 12 (2) :99-111
[22]   Optimization of Management and Processing of Big Data on a Platform for Distributed Data Storage [J].
Neric, Vedrana ;
Sarajlic, Nermin ;
Hadzic, Dulaga .
ELEKTROTEHNISKI VESTNIK, 2024, 91 (05) :272-283
[23]   A learned cost model for big data query processing [J].
Li, Yan ;
Wang, Liwei ;
Wang, Sheng ;
Sun, Yuan ;
Zheng, Bolong ;
Peng, Zhiyong .
INFORMATION SCIENCES, 2024, 670
[24]   Architectural Solution for Virtualized Processing of Big Earth Data [J].
Bica, Mihai ;
Bacu, Victor ;
Mihon, Danut ;
Gorgan, Dorian .
2014 IEEE INTERNATIONAL CONFERENCE ON INTELLIGENT COMPUTER COMMUNICATION AND PROCESSING (ICCP), 2014, :399-404
[25]   The Technique of GIS Desktop Extension [J].
Cheng Shuo ;
Xu Mingkun .
2015 INTERNATIONAL CONFERENCE ON INDUSTRIAL INFORMATICS - COMPUTING TECHNOLOGY, INTELLIGENT TECHNOLOGY, INDUSTRIAL INFORMATION INTEGRATION (ICIICII), 2015, :59-62
[26]   A Survey on Job Scheduling Algorithms in Big Data Processing [J].
Gautam, Jyoti V. ;
Prajapati, Harshadkumar B. ;
Dabhi, Vipul K. ;
Chaudhary, Sanjay .
2015 IEEE INTERNATIONAL CONFERENCE ON ELECTRICAL, COMPUTER AND COMMUNICATION TECHNOLOGIES, 2015,
[27]   Horizontal Scaling Enhancement for Optimized Big Data Processing [J].
Roy, Chandrima ;
Barua, Kashyap ;
Agarwal, Sandeep ;
Pandey, Manjusha ;
Rautaray, Siddharth Swarup .
EMERGING TECHNOLOGIES IN DATA MINING AND INFORMATION SECURITY, IEMIS 2018, VOL 1, 2019, 755 :639-649
[28]   A critical evaluation of handling uncertainty in Big Data processing [J].
Upadhyay, Ekansh .
ADVANCES IN ENGINEERING SOFTWARE, 2022, 173
[29]   Analysis and Optimization of Big-Data Stream Processing [J].
Vakilinia, Shahin ;
Zhang, Xinyao ;
Qiu, Dongyu .
2016 IEEE GLOBAL COMMUNICATIONS CONFERENCE (GLOBECOM), 2016,
[30]   Big data GIS analysis for novel approaches in building stock modelling [J].
Buffat, Rene ;
Froemelt, Andreas ;
Heeren, Niko ;
Raubal, Martin ;
Hellweg, Stefanie .
APPLIED ENERGY, 2017, 208 :277-290