On-Board-Unit Data: A Big Data Platform for Scalable Storage and Processing

被引：0
|
作者：
Buroni, Giovanni [1 ]
Le Borgne, Yann-Ael [1 ]
Bontempi, Gianluca [1 ]
Determe, Karl [2 ]
机构：
[1] Univ Libre Bruxelles, Comp Sci Dept, Machine Learning Grp, Brussels, Belgium
[2] Bruxelles Mobilite Brussel Mobiliteit, Brussels, Belgium
来源：
2018 4TH INTERNATIONAL CONFERENCE ON CLOUD COMPUTING TECHNOLOGIES AND APPLICATIONS (CLOUDTECH) | 2018年
关键词：
freight transportation; big data; lambda architecture;
D O I：
暂无
中图分类号：
TP [自动化技术、计算机技术];
学科分类号：
0812 ;
摘要：
This paper proposes and assesses a Big Data Platform for effective storage and analysis of On Board Unit (OBU) data related to the mobility of trucks in Belgium. The large volume and the streaming nature of the OBU data requires the setup of a big data platform for an efficient collection, storage and analysis. The solution relies on i) the Hadoop Distributed File System (HDFS) to store data, ii) the Apache Parquet format for data compression and columnar storage, and iii) Spark for parallel and streaming processing of data. Data replication, compression and columnar storage ensure robustness to node failure, data distribution, and faster access to data.
引用
收藏
页数：5
相关论文
共 50 条
  • [31] On Efficient Hierarchical Storage for Big Data Processing
    Krish, K. R.
    Wadhwa, Bharti
    Iqbal, M. Safdar
    Rafique, M. Mustafa
    Butt, Ali R.
    2016 16TH IEEE/ACM INTERNATIONAL SYMPOSIUM ON CLUSTER, CLOUD AND GRID COMPUTING (CCGRID), 2016, : 403 - 408
  • [32] Scalable data summarization on big data
    Feifei Li
    Suman Nath
    Distributed and Parallel Databases, 2014, 32 : 313 - 314
  • [33] Scalable data summarization on big data
    Li, Feifei
    Nath, Suman
    DISTRIBUTED AND PARALLEL DATABASES, 2014, 32 (03) : 313 - 314
  • [34] Scalable Storage Structure for Pattern Matching on Big Graph Data
    Balaji, Janani
    Sunderraman, Rajshekhar
    PROCEEDINGS 2015 IEEE INTERNATIONAL CONFERENCE ON BIG DATA, 2015, : 1848 - 1855
  • [35] A Big Data Platform for Large Scale Event Processing
    Gulisano, Vincenzo
    Jimenez-Peris, Ricardo
    Patino-Martinez, Marta
    Soriente, Claudio
    Valduriez, Patrick
    ERCIM NEWS, 2012, (89): : 32 - 33
  • [36] A Big Data Processing Platform for Medical Records in Cloud
    Yang, Chao-Tung
    Liu, Jung-Chun
    Lu, Hsin-Wen
    Yan, Yin-Zhen
    Chu, Cheng-Chung
    INTELLIGENT SYSTEMS AND APPLICATIONS (ICS 2014), 2015, 274 : 1406 - 1415
  • [37] A Design of Processing Platform for Smartphone Based on Big Data
    Zhang, Jie
    Zhang, Ke
    Zhou, Hengxin
    2015 8TH INTERNATIONAL SYMPOSIUM ON COMPUTATIONAL INTELLIGENCE AND DESIGN (ISCID), VOL 1, 2015, : 162 - 165
  • [38] Enabling Scientific Data Storage and Processing on Big-data Systems
    Biookaghazadeh, Saman
    Xu, Yiqi
    Zhou, Shujia
    Zhao, Ming
    PROCEEDINGS 2015 IEEE INTERNATIONAL CONFERENCE ON BIG DATA, 2015, : 1978 - 1984
  • [39] OpenStack Platform and its Application in Big Data Processing
    Shao, Cen
    Liang, Bo
    Wang, Feng
    Deng, Hui
    Dai, Wei
    Wei, Shoulin
    Zhang, Xiaoli
    Yuan, Zhi
    2015 8TH INTERNATIONAL CONFERENCE ON INTELLIGENT NETWORKS AND INTELLIGENT SYSTEMS (ICINIS), 2015, : 98 - 101
  • [40] Efficient query processing platform for uncertain big data
    Huang, Zhenhua
    Zhang, Jiawen
    Fang, Qiang
    International Journal of Database Theory and Application, 2015, 8 (05): : 149 - 160
12345