LARGE VECTOR SPATIAL DATA STORAGE AND QUERY PROCESSING USING CLICKHOUSE

被引:1
作者
Chen, Shuaijun [1 ]
Wang, Zhibao [1 ,2 ]
Bai, Lu [3 ]
Liu, Kunyi [1 ]
Gao, Juntao [1 ]
Zhao, Man [4 ]
Mulvenna, Maurice D. [3 ]
机构
[1] Northeast Petr Univ, Sch Comp & Informat Technol, Daqing 163318, Peoples R China
[2] Northeast Petr Univ, Bohai Rim Energy Res Inst, Qinhuangdao 066004, Hebei, Peoples R China
[3] Ulster Univ, Sch Comp, Belfast BT15, Antrim, North Ireland
[4] Qiqihar Univ, Sch Commun & Elect Engn, Qiqihar 161003, Peoples R China
来源
39TH INTERNATIONAL SYMPOSIUM ON REMOTE SENSING OF ENVIRONMENT ISRSE-39 FROM HUMAN NEEDS TO SDGS, VOL. 48-M-1 | 2023年
关键词
ClickHouse; vector spatial data; query processing; HBase; remote sensing;
D O I
10.5194/isprs-archives-XLVIII-M-1-2023-65-2023
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
The exponential growth of geospatial data resulting from the development of earth observation technology has created significant challenges for traditional relational databases. While NoSQL databases based on distributed file systems can handle massive data storage, they often struggle to cope with real-time query. Column-storage databases, on other hand, are highly effective at both storage and query processing for large-scale datasets. In this paper, we propose a spatial version of ClickHouse that leverages R-Tree indexing to enable efficient storage and real-time analysis of massive remote sensing data. ClickHouse is a column-oriented, open-source database management system designed for handling large-scale datasets. By integrating R-Tree indexing, we have created a highly efficient system for storing and querying geospatial data. To evaluate the performance of our system, we compare it with HBase, a popular distributed, NoSQL database system. Our experimental results show that ClickHouse outperforms HBase in handling spatial data queries, with a response time approximately three times faster than HBase. We attribute this performance gain to the highly efficient R-Tree indexing used in ClickHouse, which allows for fast spatial data query.
引用
收藏
页码:65 / 72
页数:8
相关论文
共 21 条
[1]   Local Geographic Information Storing and Querying using Elasticsearch [J].
Bartlett, Rebecca .
PROCEEDINGS OF THE 13TH WORKSHOP ON GEOGRAPHIC INFORMATION RETRIEVAL (GIR'19), 2019,
[2]   Big Data for Remote Sensing: Challenges and Opportunities [J].
Chi, Mingmin ;
Plaza, Antonio ;
Benediktsson, Jon Atli ;
Sun, Zhongyi ;
Shen, Jinsheng ;
Zhu, Yangyong .
PROCEEDINGS OF THE IEEE, 2016, 104 (11) :2207-2219
[3]  
CODD EF, 1970, COMMUN ACM, V13, P377, DOI 10.1145/357980.358007
[4]  
geofabrik, Papua New Guinea Data
[5]  
Ghemawat S., 2003, Operating Systems Review, V37, P29, DOI 10.1145/1165389.945450
[6]  
Guttman A., 1984, SIGMOD Record, V14, P47, DOI 10.1145/971697.602266
[7]  
Jie Zhang, 2021, 2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS, P3753, DOI 10.1109/IGARSS47720.2021.9554689
[8]  
Kucherov B, 2017, 2017 SMART CITY SYMPOSIUM PRAGUE (SCSP)
[9]   A Review of Remote Sensing for Environmental Monitoring in China [J].
Li, Jun ;
Pei, Yanqiu ;
Zhao, Shaohua ;
Xiao, Rulin ;
Sang, Xiao ;
Zhang, Chengye .
REMOTE SENSING, 2020, 12 (07)
[10]  
Liu JJ, 2014, INT CONF GEOINFORM