ScienceEarth: A Big Data Platform for Remote Sensing Data Processing

被引:37
|
作者
Xu, Chen [1 ]
Du, Xiaoping [1 ]
Yan, Zhenzhen [1 ]
Fan, Xiangtao [1 ]
机构
[1] Chinese Acad Sci, Inst Remote Sensing & Digital Earth, Key Lab Digital Earth Sci, Beijing 100094, Peoples R China
基金
中国国家自然科学基金;
关键词
big data; remote sensing data processing; distributed file system; HBase; Spark;
D O I
10.3390/rs12040607
中图分类号
X [环境科学、安全科学];
学科分类号
08 ; 0830 ;
摘要
Mass remote sensing data management and processing is currently one of the most important topics. In this study, we introduce ScienceEarth, a cluster-based data processing framework. The aim of ScienceEarth is to store, manage, and process large-scale remote sensing data in a cloud-based cluster-computing environment. The platform consists of the following three main parts: ScienceGeoData, ScienceGeoIndex, and ScienceGeoSpark. ScienceGeoData stores and manages remote sensing data. ScienceGeoIndex is an index and query system, a spatial index based on quad-tree and Hilbert curve which is combined for heterogeneous tiled remote sensing data that makes efficient data retrieval in ScienceGeoData. ScienceGeoSpark is an easy-to-use computing framework in which we use Apache Spark as the analytics engine for big remote sensing data processing. The result of tests proves that ScienceEarth can efficiently store, retrieve, and process remote sensing data. The results reveal ScienceEarth has the potential and capabilities of efficient big remote sensing data processing.
引用
收藏
页数:20
相关论文
共 50 条
  • [41] Distributed Deep Learning for Big Remote Sensing Data Processing on Apache Spark: Geological Remote Sensing Interpretation as a Case Study
    Long, Ao
    Han, Wei
    Huang, Xiaohui
    Li, Jiabao
    Wang, Yuewei
    Chen, Jia
    WEB AND BIG DATA, PT I, APWEB-WAIM 2023, 2024, 14331 : 96 - 110
  • [42] A Big Data Platform for Large Scale Event Processing
    Gulisano, Vincenzo
    Jimenez-Peris, Ricardo
    Patino-Martinez, Marta
    Soriente, Claudio
    Valduriez, Patrick
    ERCIM NEWS, 2012, (89): : 32 - 33
  • [43] A Big Data Processing Platform for Medical Records in Cloud
    Yang, Chao-Tung
    Liu, Jung-Chun
    Lu, Hsin-Wen
    Yan, Yin-Zhen
    Chu, Cheng-Chung
    INTELLIGENT SYSTEMS AND APPLICATIONS (ICS 2014), 2015, 274 : 1406 - 1415
  • [44] A Design of Processing Platform for Smartphone Based on Big Data
    Zhang, Jie
    Zhang, Ke
    Zhou, Hengxin
    2015 8TH INTERNATIONAL SYMPOSIUM ON COMPUTATIONAL INTELLIGENCE AND DESIGN (ISCID), VOL 1, 2015, : 162 - 165
  • [45] OpenStack Platform and its Application in Big Data Processing
    Shao, Cen
    Liang, Bo
    Wang, Feng
    Deng, Hui
    Dai, Wei
    Wei, Shoulin
    Zhang, Xiaoli
    Yuan, Zhi
    2015 8TH INTERNATIONAL CONFERENCE ON INTELLIGENT NETWORKS AND INTELLIGENT SYSTEMS (ICINIS), 2015, : 98 - 101
  • [46] Efficient query processing platform for uncertain big data
    Huang, Zhenhua
    Zhang, Jiawen
    Fang, Qiang
    International Journal of Database Theory and Application, 2015, 8 (05): : 149 - 160
  • [47] On-Board-Unit Data: A Big Data Platform for Scalable Storage and Processing
    Buroni, Giovanni
    Le Borgne, Yann-Ael
    Bontempi, Gianluca
    Determe, Karl
    2018 4TH INTERNATIONAL CONFERENCE ON CLOUD COMPUTING TECHNOLOGIES AND APPLICATIONS (CLOUDTECH), 2018,
  • [48] Google Earth Engine Cloud Computing Platform for Remote Sensing Big Data Applications: A Comprehensive Review
    Amani, Meisam
    Ghorbanian, Arsalan
    Ahmadi, Seyed Ali
    Kakooei, Mohammad
    Moghimi, Armin
    Mirmazloumi, S. Mohammad
    Moghaddam, Sayyed Hamed Alizadeh
    Mahdavi, Sahel
    Ghahremanloo, Masoud
    Parsian, Saeid
    Wu, Qiusheng
    Brisco, Brian
    IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING, 2020, 13 (13) : 5326 - 5350
  • [49] Cloud Computing in Remote Sensing: Big Data Remote Sensing Knowledge Discovery and Information Analysis
    Sabri, Yassine
    Bahja, Fadoua
    Siham, Aouad
    Maizate, Aberrahim
    INTERNATIONAL JOURNAL OF ADVANCED COMPUTER SCIENCE AND APPLICATIONS, 2021, 12 (05) : 888 - 895
  • [50] Multilayer Big Data Architecture for Remote Sensing in Eolic Parks
    Moguel, Enrique
    Preciado, Juan C.
    Sanchez-Figueroa, Fernando
    Preciado, Miguel A.
    Hernandez, Juan
    IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING, 2015, 8 (10) : 4714 - 4719