A Demonstration of SpatialHadoop: An Efficient MapReduce Framework for Spatial Data

被引：105

作者：

Eldawy, Ahmed ^{[1
]}

Mokbel, Mohamed F. ^{[1
]}

机构：

[1] Univ Minnesota, Dept Comp Sci & Engn, Minneapolis, MN 55455 USA

来源：

PROCEEDINGS OF THE VLDB ENDOWMENT | 2013年 / 6卷 / 12期

基金：

美国国家科学基金会;

关键词：

13;

D O I：

10.14778/2536274.2536283

中图分类号：

TP [自动化技术、计算机技术];

学科分类号：

0812 ;

摘要：

This demo presents SpatialHadoop as the first full-fledged MapReduce framework with native support for spatial data. SpatialHadoop is a comprehensive extension to Hadoop that pushes spatial data inside the core functionality of Hadoop. SpatialHadoop runs existing Hadoop programs as is, yet, it achieves order(s) of magnitude better performance than Hadoop when dealing with spatial data. SpatialHadoop employs a simple spatial high level language, a two-level spatial index structure, basic spatial components built inside the MapReduce layer, and three basic spatial operations: range queries, k-NN queries, and spatial join. Other spatial operations can be similarly deployed in SpatialHadoop. We demonstrate a real system prototype of SpatialHadoop running on an Amazon EC2 cluster against two sets of real spatial data obtained from Tiger Files and OpenStreetMap with sizes 60GB and 300GB, respectively.

引用

页码：1230 / 1233

页数：4

共 10 条

[1]

Dittrich J.-P., 2000, Proceedings of 16th International Conference on Data Engineering (Cat. No.00CB37073), P535, DOI 10.1109/ICDE.2000.839452

[2]

Ghoting A., 2011, ICDE

[3]

GUTTMAN A, 1984, SIGMOD

[4] Efficient Processing of k Nearest Neighbor Joins using MapReduce [J].

Lu, Wei ;

Shen, Yanyan ;

Chen, Su ;

Ooi, Beng Chin .

PROCEEDINGS OF THE VLDB ENDOWMENT, 2012, 5 (10) :1016-1027

[5]

Ma Q., 2009, P 1 INT WORKSHOP CLO, P9

[6] THE GRID FILE - AN ADAPTABLE, SYMMETRIC MULTIKEY FILE STRUCTURE [J].

NIEVERGELT, J ;

HINTERBERGER, H ;

SEVCIK, KC .

ACM TRANSACTIONS ON DATABASE SYSTEMS, 1984, 9 (01) :38-71

[7]

O'Malley O, 2008, TERABYTE SORT APACHE

[8]

Olston C., 2008, SIGMOD

[9]

SELLIS TK, 1987, VLDB

[10]

Zhang C., EDBT

← 1 →