Big Data Analytics using Hadoop Map Reduce Framework and Data Migration Process

被引:0
作者
Bante, Payal M. [1 ]
Rajeswari, K. [1 ]
机构
[1] Pimpri Chinchwad Coll Engn, Dept Comp, Pune, Maharashtra, India
来源
2017 INTERNATIONAL CONFERENCE ON COMPUTING, COMMUNICATION, CONTROL AND AUTOMATION (ICCUBEA) | 2017年
关键词
MySQL; NoSQL; MongoDB; HDFS; Hadoop; Big Data; Analytics; Hive; Sqoop;
D O I
暂无
中图分类号
TM [电工技术]; TN [电子技术、通信技术];
学科分类号
0808 ; 0809 ;
摘要
Database are increasing in tremendous speed, volume (terabyte to petabyte), and types (variety of Data) becoming more complex. Managing such big data has turn out to be the comprehensive challenge. To conquer this problem, Migration of Data from MySQL to NoSQL and bigdata processing performs through a programming concept identified as Hadoop MapReduce. This paper provides methodology for Migration of data from relation to NoSQL (MongoDB) database and bigdata analytics using Hadoop Map Reduce framework above Hadoop Distributed File System (HDFS). Also Sqoop is used for migrating data from Relational Database to Hadoop for analytics process. Hive is introduced for migrate analyzed data from Hadoop to MongoDB. All Experiments are performed on Four different datasets-Loan Database, Connect 4 dataset, Lenses dataset and player tennis dataset form UCI Repository and KAGGLE repository.
引用
收藏
页数:5
相关论文
共 10 条
[1]  
Dai Wei., 2014, International Journal of Database Theory and Application, V7, P49, DOI DOI 10.14257/IJDTA.2014.7.1.05
[2]  
GAIOSO R., 2007, THESIS
[3]  
Gilbert Seth, 2002, SIGACT NEWS, V33, P5159
[4]  
Han J, 2012, MOR KAUF D, P1
[5]  
Ierusalimschy R., 1996, SOFTWARE PRACT EXPER, V26
[6]  
Liao Ying-Ti, 2011, FUTURE GENER COMP SY, V10, P54
[7]  
Lotfy Ayman E., 2016, J KING SAUD COMPUTER, V28
[8]  
Mohmad Hesham Mohamed, 2011, EGYPTIAN INFORM J, V12, P7382
[9]  
Padhy R. P., 2011, INT ADV ENG SCI TECH, V11
[10]  
Vale Fernando, 2015, ICCS INT C COMP SCI, V21, P2013