HaoLap: A Hadoop based OLAP system for big data

被引：30
|
作者：
Song, Jie [1 ]
Guo, Chaopeng [1 ]
Wang, Zhi [1 ]
Zhang, Yichan [1 ]
Yu, Ge [2 ]
Pierson, Jean-Marc [3 ]
机构：
[1] Northeastern Univ, Software Coll, Shenyang 110819, Peoples R China
[2] Northeastern Univ, Sch Informat & Engn, Shenyang 110819, Peoples R China
[3] Univ Toulouse 3, Lab IRIT, F-31062 Toulouse, France
来源：
JOURNAL OF SYSTEMS AND SOFTWARE | 2015年 / 102卷
基金：
新加坡国家研究基金会; 中国国家自然科学基金;
关键词：
Cloud data warehouse; Multidimensional data model; MapReduce;
D O I：
10.1016/j.jss.2014.09.024
中图分类号：
TP31 [计算机软件];
学科分类号：
081202 ; 0835 ;
摘要：
In recent years, facing information explosion, industry and academia have adopted distributed file system and MapReduce programming model to address new challenges the big data has brought. Based on these technologies, this paper presents HaoLap (Hadoop based oLap), an OLAP (OnLine Analytical Processing) system for big data. Drawing on the experience of Multidimensional OLAP (MOLAP), HaoLap adopts the specified multidimensional model to map the dimensions and the measures; the dimension coding and traverse algorithm to achieve the roll up operation on dimension hierarchy; the partition and linearization algorithm to store dimensions and measures; the chunk selection algorithm to optimize OLAP performance; and MapReduce to execute OLAP. The paper illustrates the key techniques of HaoLap including system architecture, dimension definition, dimension coding and traversing, partition, data storage, OLAP and data loading algorithm. We evaluated HaoLap on a real application and compared it with Hive, HadoopDB, HBaseLattice, and Olap4Cloud. The experiment results show that HaoLap boost the efficiency of data loading, and has a great advantage in the OLAP performance of the data set size and query complexity, and meanwhile HaoLap also completely support dimension operations. (C) 2014 Elsevier Inc. All rights reserved.
引用
收藏
页码：167 / 181
页数：15
相关论文
共 50 条
  • [41] Improved CURE Clustering for Big Data using Hadoop and Mapreduce
    Lathiya, Piyush
    Rani, Rinkle
    2016 INTERNATIONAL CONFERENCE ON INVENTIVE COMPUTATION TECHNOLOGIES (ICICT), VOL 3, 2015, : 241 - 245
  • [42] An Approach to Enhance the Performance of Hadoop MapReduce Framework for Big Data
    Chandra, Subhash
    Motwani, Deepak
    2016 INTERNATIONAL CONFERENCE ON MICRO-ELECTRONICS AND TELECOMMUNICATION ENGINEERING (ICMETE), 2016, : 178 - 182
  • [43] The Big Data Analysis of Two-degree Contacts on Hadoop
    Wu, Hao
    Li, Xiongfei
    PROCEEDINGS OF THE 2016 INTERNATIONAL CONFERENCE ON COMPUTER SCIENCE AND ELECTRONIC TECHNOLOGY, 2016, 48 : 167 - 171
  • [44] Big Data representation for Grade Analysis Through Hadoop Framework
    Verma, Chitresh
    Pandey, Rajiv
    2016 6TH INTERNATIONAL CONFERENCE - CLOUD SYSTEM AND BIG DATA ENGINEERING (CONFLUENCE), 2016, : 312 - 315
  • [45] Budget Constraint Scheduler for Big Data Using Hadoop MapReduce
    Vinutha D.C.
    Raju G.T.
    SN Computer Science, 2021, 2 (4)
  • [46] Addressing Big Data Problem Using Hadoop and Map Reduce
    Patel, Aditya B.
    Birla, Manashvi
    Nair, Ushma
    3RD NIRMA UNIVERSITY INTERNATIONAL CONFERENCE ON ENGINEERING (NUICONE 2012), 2012,
  • [47] Big Data: Hadoop framework vulnerabilities, security issues and attacks
    Bhathal, Gurjit Singh
    Singh, Amardeep
    ARRAY, 2019, 1-2
  • [48] HdK-Means: Hadoop Based Parallel K-Means Clustering for Big Data
    Bandyopadhyay, Soumyendu Sekhar
    Halder, Anup Kumar
    Chatterjee, Piyali
    Nasipuri, Mita
    Basu, Subhadip
    2017 IEEE CALCUTTA CONFERENCE (CALCON), 2017, : 452 - 456
  • [49] High Performance and Fault Tolerant Distributed File System for Big Data Storage and Processing using Hadoop
    Sivaraman, E.
    Manickachezian, R.
    2014 INTERNATIONAL CONFERENCE ON INTELLIGENT COMPUTING APPLICATIONS (ICICA 2014), 2014, : 32 - 36
  • [50] A COMPARATIVE ANALYSIS OF CONVENTIONAL HADOOP WITH PROPOSED CLOUD ENABLED HADOOP FRAMEWORK FOR SPATIAL BIG DATA PROCESSING
    Tripathi, A. K.
    Agrawal, S.
    Gupta, R. D.
    ISPRS TC V MID-TERM SYMPOSIUM GEOSPATIAL TECHNOLOGY - PIXEL TO PEOPLE, 2018, 4-5 : 425 - 430
12345