An Optimized Distributed OLAP System for Big Data

被引：0

作者：

Chen, Wenhao ^{[1
]}

Wang, Haoxiang ^{[1
]}

Zhang, Xingming ^{[1
]}

Lin, Qidi ^{[1
]}

机构：

[1] South China Univ Technol, Sch Comp Sci & Engn, Guangzhou 510006, Guangdong, Peoples R China

来源：

2017 2ND IEEE INTERNATIONAL CONFERENCE ON COMPUTATIONAL INTELLIGENCE AND APPLICATIONS (ICCIA) | 2017年

关键词：

big data; decision making; OLAP;

D O I：

暂无

中图分类号：

TP18 [人工智能理论];

学科分类号：

081104 ; 0812 ; 0835 ; 1405 ;

摘要：

To solve the problems of heterogeneous data types and large amount of calculation in making decision for big data, an optimized distributed OLAP system for big data is proposed in this paper. The system provides data acquisition for different data sources, and supports two types of OLAP engines, Impala and Kylin. First of all, the architecture of the system is proposed, consisting of four modules, data acquisition, data storage, OLAP analysis and data visualization, and the specific implementation of each module is descripted in great detail. Then the optimization of the system is put forward, which is automatic metadata configuration and the cache for OLAP query. Finally, the performance test of the system is conduct to demonstrate that the efficiency of the system is significantly better than the traditional solution.

引用

页码：36 / 40

页数：5

共 16 条

[1] On-line analytical processing in distributed data warehouses
Albrecht, J
Lehner, W
[J]. IDEAS 98 - INTERNATIONAL DATABASE ENGINEERING AND APPLICATIONS SYMPOSIUM, PROCEEDINGS, 1998, : 78 - 85
[2] [Anonymous], 2011, 14 INT C EXT DAT TEC, DOI DOI 10.1145/1951365.1951432
[3] [Anonymous], 1983, Signals and Systems
[4] Balog M, 2016, THE MONDRIAN KERNEL
[5] Chen Q, 2012, Distributed caching and analysis system and method: US, Patent No. [US 20120303901 A1, 20120303901]
[6] A Cloud-based Framework for Supporting Effective and Efficient OLAP in Big Data Environments
Cuzzocrea, Alfredo
Moussa, Rim
[J]. 2014 14TH IEEE/ACM INTERNATIONAL SYMPOSIUM ON CLUSTER, CLOUD AND GRID COMPUTING (CCGRID), 2014, : 680 - 684
[7] Scalable real-time OLAP on cloud architectures
Dehne, F.
Kong, Q.
Rau-Chaplin, A.
Zaboli, H.
Zhou, R.
[J]. JOURNAL OF PARALLEL AND DISTRIBUTED COMPUTING, 2015, 79-80 : 31 - 41
[8] A Cloud-based Efficient On-line Analytical Processing System with Inverted Data Model
Huang, Sheng-Wei
Shieh, Ce-Kuen
Liao, Che-Ching
Chiu, Chui-Ming
Tsai, Ming-Fong
Chen, Lien-Wu
[J]. PROCEEDINGS OF THE 11TH EAI INTERNATIONAL CONFERENCE ON HETEROGENEOUS NETWORKING FOR QUALITY, RELIABILITY, SECURITY AND ROBUSTNESS, 2015, : 341 - 345
[9] Kornacker M., 2012, Cloudera Impala: Real-Time Queries in Apache Hadoop, For Real
[10] Li Jian, 2010, 2010 Seventh International Conference on Fuzzy Systems and Knowledge Discovery (FSKD 2010), P2570, DOI 10.1109/FSKD.2010.5569837

← 1 2 →