Multimodal deep learning framework for vessel trajectory prediction

Maritime ship trajectory prediction is an essential but challenging research topic in intelligent maritime traffic. It has garnered growing attention due to developments in deep learning methods. Although deep learning networks have been employed in the prediction task of single-modal automatic identification system (AIS) data, their performance inevitably faces bottlenecks in complex scenes that require reliable prediction due to the limitations of marine environment factors. In this study, we propose a solution to this problem by designing a multimodal deep learning trajectory prediction (MDL-TP) framework. Timestamps and shortest distances were used to fuse marine ship spatiotemporal and environmental data. We also designed an extraction and fusion network architecture based on the multimodal data. Specifically, five trajectory prediction models were designed and implemented using a unified MDL-TP framework. Finally, we verified the effectiveness and superiority of the MDL-TP framework on actual AIS and maritime environment datasets along the West Coast of the United States. The five models provided by our MDL-TP framework have an average accuracy improvement of 40.18% in the MAE evaluation metric compared to all baseline comparison models. Moreover, qualitative analysis and ablation experiments proved the superiority of our framework.