ST-MAN: Spatio-Temporal Multimodal Attention Network for Traffic Prediction

Traffic prediction is an essential part of Intelligent Transportation System (ITS). Existing work typically use unimodal traffic data, combining with road network graph or external factors (e.g., weather, POIs) for prediction. However, in real traffic systems multi-modal traffic data are collected from one or more co-located sensors, and data of non-target modality are not fully utilized by existing work. To overcome this limitation, we utilize multimodal traffic data to improve target prediction tasks. We propose a novel Spatio-Temporal Multimodal Attention Network (ST-MAN) for traffic prediction. Firstly, we design a cross-modal attention mechanism to learn dynamic inter-modal correlations. Secondly, we propose a compact yet effective multimodal fusion framework to exploit both the inter-modal and intra-modal correlations. Thirdly, a refined spatio-temporal embedding mechanism is designed to feed in more implicit information. Extensive experiments on three realworld datasets show that ST-MAN not only outperforms state-of-the-art methods in all aspects, but also has high computational efficiency. Moreover, the framework is easily generalized to include more data modalities.