Iron and Steel Enterprises Big Data Visualization Analysis Based on Spark

被引:1
作者
Ban, Xiaojuan [1 ]
Wang, Ben [1 ]
Cheng, Changxin [1 ]
Taghzouit, Salah [1 ]
机构
[1] Univ Sci & Technol Beijing, Beijing 100083, Peoples R China
来源
COOPERATIVE DESIGN, VISUALIZATION, AND ENGINEERING: 15TH INTERNATIONAL CONFERENCE, CDVE 2018 | 2018年 / 11151卷
基金
中国博士后科学基金; 美国国家科学基金会;
关键词
Visualization; Industrial intelligence; Data analysis; ECharts; Spark; MAPREDUCE;
D O I
10.1007/978-3-030-00560-3_39
中图分类号
TP301 [理论、方法];
学科分类号
081202 ;
摘要
Given the difficulty of browsing and analyzing big data via web browsers, using Spark technology, we took the big-data analysis of steel companies as an example to propose a framework for big-data visualization technology. We used HDFS for data storage, Spark for data analysis, Django for web systems, and ECharts for data visualization, ultimately providing a complete visualization solution. Using visualization, we realized price forecasting, sales analysis and production process quality traceability, help enterprises to make decisions and provide support for technological process improvement.
引用
收藏
页码:280 / 286
页数:7
相关论文
共 11 条
[1]  
[Anonymous], 2010, HOTCLOUD
[2]  
[Anonymous], 2012, RESILIENT DISTRIBUTE
[3]  
Chen HC, 2012, MIS QUART, V36, P1165
[4]  
Dean J, 2004, USENIX ASSOCIATION PROCEEDINGS OF THE SIXTH SYMPOSIUM ON OPERATING SYSTEMS DESIGN AND IMPLEMENTATION (OSDE '04), P137
[5]  
Fiaz AS., 2016, International Journal of Applied Engineering Research, V11, P2801
[6]  
Franklin M., 2013, BERKELEY DATA ANAL S
[7]  
Holovaty Adrian, 2009, The Definitive Guide to Django: Web Development Done Right
[8]  
[刘强 Liu Qiang], 2016, [自动化学报, Acta Automatica Sinica], V42, P161
[9]   Clash of the Titans: MapReduce vs. Spark for Large Scale Data Analytics [J].
Shi, Juwei ;
Qiu, Yunjie ;
Minhas, Umar Farooq ;
Jiao, Limei ;
Wang, Chen ;
Reinwald, Berthold ;
Ozcan, Fatma .
PROCEEDINGS OF THE VLDB ENDOWMENT, 2015, 8 (13) :2110-2121
[10]  
Shvachko K., 2010, HADOOP DISTRIBUTED F