Distracted driving recognition based on functional connectivity analysis between physiological signals and perinasal perspiration index

Automatic detection of distracted driving is essential to ensure safety of drivers. In this paper, a novel set of features were extracted from thermal and physiological signals in order to detect and recognize distraction of drivers. Thermal video data which measured the temperature of different areas of the face, heart rate, breathing rate and behavioural signals were used while various types of distractions including cognitive, emotional and sensory-motor were applied to the subjects. The proposed discriminator features were extracted by different functional connectivity methods between the perinasal perspiration extracted from thermal images of the face and physiological variables of heart rate and breathing rate. After feature extraction, binary classification methods were applied to detect the distractions. The results showed that using functional connectivity features significantly increased the accuracy of distraction detection system (up to 99.16% with a highly acceptable F1 score of 0.99). Hence, the proposed model significantly improved (p-value < 0.001) the detection of driver's distraction compared to previous studies. Functional connectivity between perinasal perspiration and breathing signals was the most informative feature for distraction detection purpose. This feature set explained almost all modulations in physiological changes by distractions. For the first time, we designed a system capable of recognizing types of distractions including sensory-motor, cognitive, emotional distractions and also normal state. We used the same feature set to recognize different types of distractions by using three-class and four-class classifiers. The suggested methods distinguished three types of distractions with the best accuracy of 81.94% related to cognitive, sensory-motor distraction and no-distraction states. We also tried to discriminate two types of cognitive distractions, analytic and mathematical distractions. The recognition system classified two types of cognitive distractions with an accuracy of 91.78%. The results suggest that there is important and complementary information in the connectivity between facial temperature signals and physiological variables for distraction detection and recognition.