Unmanned Aerial Vehicles anomaly detection model based on sensor information fusion and hybrid multimodal neural network

The use of Unmanned Aerial Vehicle (UAV) in various industries is increasing, which places higher requirements on the reliability of UAV. One of the ways to ensure the safety of UAV flights is by detecting anomalies in their flight. However, traditional UAV anomaly detection models have some shortcomings. First, they fail to integrate data from multiple sensors across time and frequency domains, hampering the anomaly detection model's ability to accurately assess the UAV's status. Second, they apply the same prediction error loss to all classes, which result in excessive false positives in some key classes. Finally, most of them used unimodal classification models to process data from multiple heterogeneous sensors, which makes it difficult for the models to extract targeted features. This paper proposes a UAV anomaly detection model based on sensor information fusion and hybrid multimodal neural network (IF-HMNN). Firstly, facilitated by the newly devised Multi-source Heterogeneous UAV Sensor Information Alignment algorithm (MHSIA), IF-HMNN can realize information fusion from multiple sensors. Secondly, a classes weight assignment mechanism is designed to increase the IF-HMNN's focus on key classes. Finally, the neural networks of two modalities are trained separately according to different timefrequency domain features, and their classification outcomes are amalgamated through a hybrid soft voting mechanism. Experimental results show that IF-HMNN achieves accuracy of 0.99, 0.9991, and 0.9967 on three datasets respectively. The accuracy of IF-HMNN model on the test set is about 2 %-3 % higher than similar models. We will publish our code as well as the dataset here: https://github.com/FishLuYu/IF-HMNN.