Multi-feature fusion prediction of fatigue driving based on improved optical flow algorithm

To predict whether a driver is fatigued, a fatigue prediction algorithm based on the fusion of improved optical flow features and microfeatures was proposed. By improving the Main Directional Mean Optical-flow (MDMO) algorithm to extract facial microfeatures, the spatial and temporal features of facial micro-expressions were obtained without loss of feature information. Simultaneously, a remote photoplethysmography-based physiological feature detection algorithm was applied to extract the facial region of interest of the driver. The algorithm improves the convenience and accuracy of heart rate variability signal extraction. Then, the physiological characteristics and micro-expression characteristics were input to the long short-term memory to obtain the corresponding timing characteristics. Finally, the two time-series features were fused to predict the fatigue state of the driver. The experiment used the CASME II video face database which contains a large number of spontaneous and dynamic micro-expressions, the self-built 20 face video database, and the actual detection items in a fatigue state for 1 min each. The experimental results show that the improved MDMO optical flow algorithm is accurate for micro-expression recognition, with a rate of 75.2%, which is 7.83% higher than that of the traditional optical flow algorithm. In the case of microfeature fusion, the accuracy of driver fatigue prediction is as high as 95.24%, showing acceptable results in the prediction of driver fatigue.