Driver Distraction Behavior Detection Framework Based on the DWPose Model, Kalman Filtering, and Multi-Transformer

Driver distraction behavior recognition is crucial for improving driving safety. Traditional end-to-end driver distraction detection models are susceptible to factors such as the driving environment, the in-vehicle background, and the driver characteristics, which leads to the low performance of the models in cross-dataset testing. To address this problem, this paper proposes a driver distraction detection framework based on the Distillation for Whole-body Pose estimators (DWPose), Kalman filtering, and Multi-Transformer (DKT). DKT consists of a feature extraction calibration module and a distraction behavior recognition module. In the feature extraction calibration module, a pre-trained DWPose model is used to extract the facial, hand, and body keypoints of the driver, and improved Kalman filtering algorithm is applied for tracking and correction keypoint sequences. In the distraction behavior recognition module, a Multi-Transformer model is used to model the relationships between keypoint sequences and the distraction behaviors of various parts of the human body. The model can extract features from the action-behavior process of different body parts and perform weighted fusion to obtain the final distraction behavior category. The results of experiments show that DKT can accurately recognize the distraction behaviors of drivers, and maintains high cross-test accuracy under the State Farm Driver 2 (SFD2) and 100-Driver datasets, thereby demonstrating its high generalization performance. Specifically, when trained using the 100-Driver dataset, the mAcc of DKT on the 100-Driver and SFD2 test sets are respectively 98.03% and 73.88%, thus exhibiting respective improvements of 1.38% and 19.56%, as compared to the MobileNetV2-CA model. The results verify that the proposed DKT is an advanced model in the field of driver distraction recognition and can provide technical support for driver distraction warning.