ScienceEarth: A Big Data Platform for Remote Sensing Data Processing

被引：37
|
作者：
Xu, Chen [1 ]
Du, Xiaoping [1 ]
Yan, Zhenzhen [1 ]
Fan, Xiangtao [1 ]
机构：
[1] Chinese Acad Sci, Inst Remote Sensing & Digital Earth, Key Lab Digital Earth Sci, Beijing 100094, Peoples R China
来源：
REMOTE SENSING | 2020年 / 12卷 / 04期
基金：
中国国家自然科学基金;
关键词：
big data; remote sensing data processing; distributed file system; HBase; Spark;
D O I：
10.3390/rs12040607
中图分类号：
X [环境科学、安全科学];
学科分类号：
08 ; 0830 ;
摘要：
Mass remote sensing data management and processing is currently one of the most important topics. In this study, we introduce ScienceEarth, a cluster-based data processing framework. The aim of ScienceEarth is to store, manage, and process large-scale remote sensing data in a cloud-based cluster-computing environment. The platform consists of the following three main parts: ScienceGeoData, ScienceGeoIndex, and ScienceGeoSpark. ScienceGeoData stores and manages remote sensing data. ScienceGeoIndex is an index and query system, a spatial index based on quad-tree and Hilbert curve which is combined for heterogeneous tiled remote sensing data that makes efficient data retrieval in ScienceGeoData. ScienceGeoSpark is an easy-to-use computing framework in which we use Apache Spark as the analytics engine for big remote sensing data processing. The result of tests proves that ScienceEarth can efficiently store, retrieve, and process remote sensing data. The results reveal ScienceEarth has the potential and capabilities of efficient big remote sensing data processing.
引用
收藏
页数：20
相关论文
共 50 条
  • [1] A Spark-Based Big Data Platform for Massive Remote Sensing Data Processing
    Sun, Zhongyi
    Chen, Fengke
    Chi, Mingmin
    Zhu, Yangyong
    DATA SCIENCE, 2015, 9208 : 120 - 126
  • [2] BIG DATA PROCESSING USING HPC FOR REMOTE SENSING DISASTER DATA
    Bhangale, Ujwala M.
    Kurte, Kuldeep R.
    Durbha, Surya S.
    King, Roger L.
    Younan, Nicolas H.
    2016 IEEE INTERNATIONAL GEOSCIENCE AND REMOTE SENSING SYMPOSIUM (IGARSS), 2016, : 5894 - 5897
  • [3] Cloud Computing in Remote Sensing : High Performance Remote Sensing Data Processing in a Big data Environment
    Sabri, Y.
    Aouad, S.
    INTERNATIONAL JOURNAL OF COMPUTERS COMMUNICATIONS & CONTROL, 2021, 16 (06)
  • [4] On-Demand Processing for Remote Sensing Big Data Analysis
    Huang, Zhenchun
    Zhong, Anrun
    Li, Guoqing
    2017 15TH IEEE INTERNATIONAL SYMPOSIUM ON PARALLEL AND DISTRIBUTED PROCESSING WITH APPLICATIONS AND 2017 16TH IEEE INTERNATIONAL CONFERENCE ON UBIQUITOUS COMPUTING AND COMMUNICATIONS (ISPA/IUCC 2017), 2017, : 1241 - 1245
  • [5] Big Data Sensing Information Processing Platform for Intelligent Traffic
    Tang, Jinpeng
    Li, Linglin
    ADVANCES IN COMPUTERS, ELECTRONICS AND MECHATRONICS, 2014, 667 : 324 - 327
  • [6] Remote sensing data parallel processing base on cloud platform
    Wei Haitao
    Du Yunyan
    Zhang Chunjin
    Wang Xin
    2011 INTERNATIONAL CONFERENCE ON PHOTONICS, 3D-IMAGING, AND VISUALIZATION, 2011, 8205
  • [7] A service-oriented framework for remote sensing big data processing
    Enayati, Roohollah
    Ravanmehr, Reza
    Aghazarian, Vahe
    EARTH SCIENCE INFORMATICS, 2023, 16 (01) : 591 - 616
  • [8] A service-oriented framework for remote sensing big data processing
    Roohollah Enayati
    Reza Ravanmehr
    Vahe Aghazarian
    Earth Science Informatics, 2023, 16 : 591 - 616
  • [9] A Comparison of Big Remote Sensing Data Processing with Hadoop MapReduce and Spark
    Chebbi, I.
    Boulila, W.
    Mellouli, N.
    Lamolle, M.
    Farah, I. R.
    2018 4TH INTERNATIONAL CONFERENCE ON ADVANCED TECHNOLOGIES FOR SIGNAL AND IMAGE PROCESSING (ATSIP), 2018,
  • [10] Research of Big Data Processing Platform
    Liu, Xiangju
    GREEN POWER, MATERIALS AND MANUFACTURING TECHNOLOGY AND APPLICATIONS III, PTS 1 AND 2, 2014, 484-485 : 922 - 926
12345