Next location prediction using transformers

被引：0
作者：
Henouda S.E. [1 ]
Laallam F.Z. [1 ]
Kazar O. [2 ,3 ]
Sassi A. [4 ,5 ]
机构：
[1] LINATI Laboratory, Department of Computer Science, Kasdi Merbah University, Ouargla
[2] Smart Computer Science Laboratory (LINFI), Computer Science Department, University of Biskra
[3] Department of Information Systems and Security, College of Information Technology, United Arab Emirate University
[4] Department of Computer Science, Mohamed Khider University, Biskra
[5] Department of computer Science, L’arbi Ben Mhidi University, Oum El Bouaghi
来源：
International Journal of Business Intelligence and Data Mining | 2022年 / 21卷 / 02期
关键词：
big data; deep learning; machine learning; mobility traces; neural networks; next location prediction; transformer; Wi-Fi;
D O I：
10.1504/IJBIDM.2022.124851
中图分类号：
学科分类号：
摘要：
This work seeks to solve the next location prediction problem of mobile users. Chiefly, we focus on ROBERTA architecture (robustly optimised BERT approach) in order to build a next location prediction model through the use of a subset of a large real mobility trace database. The latter was made available to the public through the CRAWDAD project. ROBERTA, which is a well-known model in natural language processing (NLP), works intentionally on predicting hidden sections of text based on language masking strategy. The current paper follows a similar architecture as ROBERTA and proposes a new combination of Bertwordpiece tokeniser and ROBERTA for location prediction that we call WP-BERTA. The results demonstrated that our proposed model WP-BERTA outperformed the state-of-the-art models. They also indicated that the proposed model provided a significant improvement in the next location prediction accuracy compared to the state-of-the-art models. We particularly revealed that WP-BERTA outperformed Markovian models, support vector machine (SVM), convolutional neural networks (CNNs), and long short-term memory networks (LSTMs). Copyright © 2022 Inderscience Enterprises Ltd.
引用
收藏
页码：247 / 263
页数：16
相关论文
共 35 条
  • [1] Al-Molegi A., Jabreel M., Ghaleb B., STF-RNN: space time features-based recurrent neural network for predicting people next location, 2016 IEEE Symposium Series on Computational Intelligence, SSCI 2016, (2017)
  • [2] Asahara A., Maruyama K., Sato A., Seto K., Pedestrian-movement prediction based on mixed Markov-chain model, GIS: Proceedings of the ACM International Symposium on Advances in Geographic Information Systems, pp. 25-33, (2011)
  • [3] Baumann P., Kleiminger W., Santini S., The influence of temporal and spatial features on the performance of next-place prediction algorithms, UbiComp 2013 – Proceedings of the 2013 ACM International Joint Conference on Pervasive and Ubiquitous Computing, pp. 449-458, (2013)
  • [4] Chung J., Gulcehre C., Cho K., Bengio Y., Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling, (2014)
  • [5] Crivellari A., Beinat E., LSTM-based deep learning model for predicting individual mobility traces of short-term foreign tourists, Sustainability (Switzerland), 12, 1, (2020)
  • [6] Devlin J., Chang M.W., Lee K., Toutanova K., BERT: pre-training of deep bidirectional transformers for language understanding, NAACL HLT 2019 – 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies – Proceedings of the Conference, 1, pp. 4171-4186, (2019)
  • [7] Dong L., Yang N., Wang W., Wei F., Liu X., Wang Y., Gao J., Zhou M., Hon H.W., Unified language model pre-training for natural language understanding and generation, Advances in Neural Information Processing Systems (NeurIPS), 32, (2019)
  • [8] Gambs S., Killijian M.O., Del Prado Cortez M.N., Show me how you move and I will tell you who you are, Proceedings of the 3rd ACM SIGSPATIAL International Workshop on Security and Privacy in GIS and LBS, SPRINGL 2010, pp. 34-41, (2010)
  • [9] Gambs S., Killijian M.O., Del Prado Cortez M.N., Next place prediction using mobility Markov chains, Proceedings of the 1st Workshop on Measurement, Privacy, and Mobility, MPM’12, pp. 0-5, (2012)
  • [10] Geng S., Gao P., Fu Z., Zhang Y., RomeBERT: Robust Training of Multi-Exit BERT, (2021)
1234