A Novel Spatial-Temporal Graph for Skeleton-based Driver Action Recognition

Driver irregular operations have long been recognized as the main contributing factors in traffic accidents. Therefore, early and accurate identification of driver actions can help mitigate the effects of potentially unsafe driving behaviors and avoid possible accidents. This paper studies skeleton-based driver action recognition using spatial-temporal graph and genetically-weighted algorithm. We first determine the body pose of the driver by selectively sampling the informative frames in a video, which includes driver joints with high confidence scores and joints position coordinate obtained in pose estimation algorithm. Afterward, we use the skeleton-based graph in the spatial-temporal domains to estimate driver's joints position, and these dynamic skeletons of the captured poses are used as inputs to the Graph Convolutional Networks (GCNs). Finally, the genetically-weighted algorithm is used to vote for the joint point with the highest score of association and significant position change, and we adopt the correlation between the sensory input and the impending driver behavior to determine the driver's detailed actions. We experiment in the Kinetics, NTU-RGB+D and StateFarm datasets. The results show that our method has high accuracy and strong generalization capability.