On-Board-Unit Data: A Big Data Platform for Scalable Storage and Processing

被引：0
|
作者：
Buroni, Giovanni [1 ]
Le Borgne, Yann-Ael [1 ]
Bontempi, Gianluca [1 ]
Determe, Karl [2 ]
机构：
[1] Univ Libre Bruxelles, Comp Sci Dept, Machine Learning Grp, Brussels, Belgium
[2] Bruxelles Mobilite Brussel Mobiliteit, Brussels, Belgium
来源：
2018 4TH INTERNATIONAL CONFERENCE ON CLOUD COMPUTING TECHNOLOGIES AND APPLICATIONS (CLOUDTECH) | 2018年
关键词：
freight transportation; big data; lambda architecture;
D O I：
暂无
中图分类号：
TP [自动化技术、计算机技术];
学科分类号：
0812 ;
摘要：
This paper proposes and assesses a Big Data Platform for effective storage and analysis of On Board Unit (OBU) data related to the mobility of trucks in Belgium. The large volume and the streaming nature of the OBU data requires the setup of a big data platform for an efficient collection, storage and analysis. The solution relies on i) the Hadoop Distributed File System (HDFS) to store data, ii) the Apache Parquet format for data compression and columnar storage, and iii) Spark for parallel and streaming processing of data. Data replication, compression and columnar storage ensure robustness to node failure, data distribution, and faster access to data.
引用
收藏
页数：5
相关论文
共 50 条
  • [1] Cluster Analysis of On-Board-Unit Truck Big Data from the Brussels Capital Region
    Buroni, Giovanni
    Yann-Al Le Borgne
    Bontempi, Gianluca
    Determe, Karl
    2018 21ST INTERNATIONAL CONFERENCE ON INTELLIGENT TRANSPORTATION SYSTEMS (ITSC), 2018, : 2074 - 2079
  • [2] Optimization of Management and Processing of Big Data on a Platform for Distributed Data Storage
    Nerić, Vedrana
    Sarajlić, Nermin
    Hadžić, Đulaga
    Elektrotehniski Vestnik/Electrotechnical Review, 2024, 91 (05): : 272 - 283
  • [3] Optimization of Management and Processing of Big Data on a Platform for Distributed Data Storage
    Neric, Vedrana
    Sarajlic, Nermin
    Hadzic, Dulaga
    ELEKTROTEHNISKI VESTNIK, 2024, 91 (05): : 272 - 283
  • [4] Computational storage: an efficient and scalable platform for big data and HPC applications
    Torabzadehkashi, Mahdi
    Rezaei, Siavash
    HeydariGorji, Ali
    Bobarshad, Hosein
    Alves, Vladimir
    Bagherzadeh, Nader
    JOURNAL OF BIG DATA, 2019, 6 (01)
  • [5] Computational storage: an efficient and scalable platform for big data and HPC applications
    Mahdi Torabzadehkashi
    Siavash Rezaei
    Ali HeydariGorji
    Hosein Bobarshad
    Vladimir Alves
    Nader Bagherzadeh
    Journal of Big Data, 6
  • [6] Data Storage Adapter in Big Data Platform
    Minh Chau Nguyen
    Won, Hee Sun
    2015 8TH INTERNATIONAL CONFERENCE ON DATABASE THEORY AND APPLICATION (DTA), 2015, : 6 - 9
  • [7] Catalina: In-Storage Processing Acceleration for Scalable Big Data Analytics
    Torabzadehkashi, Mahdi
    Rezaei, Siavash
    Heydarigorji, Ali
    Bobarshad, Hosein
    Alves, Vladimir
    Bagherzadeh, Nader
    2019 27TH EUROMICRO INTERNATIONAL CONFERENCE ON PARALLEL, DISTRIBUTED AND NETWORK-BASED PROCESSING (PDP), 2019, : 430 - 437
  • [8] Clouds for scalable Big Data processing
    Trunfio, Paolo
    Vlassov, Vladimir
    INTERNATIONAL JOURNAL OF PARALLEL EMERGENT AND DISTRIBUTED SYSTEMS, 2019, 34 (06) : 629 - 631
  • [9] AN EFFECTIVE AND SCALABLE DATA MODELING FOR ENTERPRISE BIG DATA PLATFORM
    Patel, Jayesh
    2019 IEEE INTERNATIONAL CONFERENCE ON BIG DATA (BIG DATA), 2019, : 2691 - 2697
  • [10] Design of a Scalable Data Stream Channel for Big Data Processing
    Lee, Yong-Ju
    Lee, Myungcheol
    Lee, Mi-Young
    Hur, Sung Jin
    Min, Okgee
    2015 17TH INTERNATIONAL CONFERENCE ON ADVANCED COMMUNICATION TECHNOLOGY (ICACT), 2015, : 556 - 559
12345