Robust Pedestrian Crossing Intention Prediction via Uncertainty-Guided Transformer Ensemble Network for Autonomous Driving

Predicting pedestrian crossing behavior is becoming increasingly important for autonomous driving vehicles, especially in the scene of urban transport. Most of the previous methods concentrate on feature-level fusion that integrates various types of input data, without considering the prediction of each individual input. In order to overcome this defect, this article proposes an uncertainty-guided transformer ensemble network (UTENet) that explores the merits of both feature-level and decision-level in a unified framework. The proposed model takes only the pedestrian bounding box (bbox) and ego-vehicle velocity as input. First, for each input, we apply the self-attention mechanism to model the intramodal correlation and aggregate the correlated features at different moments. Then, we put forward a cross-modal attention-based fusion module to capture the intramodal relationships between two inputs so that a more comprehensive representation related to crossing intention can be generated. Finally, we design an uncertainty-based ensemble strategy for decision-level fusion, thus remedying the drawback of individual prediction and enhancing the robustness. The results of the experiment on the real-world benchmark dataset verify that our model can predict pedestrian crossing behavior using less modal information while achieving performance that is comparable to or even better than the methods relying on more inputs. Extensive ablation studies are also provided to verify the effectiveness of our model components.