A Real-Time Fatigue Driving Recognition Method Incorporating Contextual Features and Two Fusion Levels

Though experimental results have shown a strong correlation between contextual features and the driver's fatigue state, contextual features have been applied only offline to evaluate a driver's fatigue state. This paper identifies three of the most effective contextual features, i.e., continuous driving time, sleep duration time, and current time, to facilitate the real-time (online) recognition of fatigue state. By applying gray relational analysis, the three contextual features, together with the most effective facial and vehicle behavior features, are introduced in a two-level fusion structure to improve fatigue driving recognition. In the first level of fusion, labeled the feature-level fusion, three separate multiclass support vector machine (MCSVM) classifiers are used for the three feature sources, i.e., contextual features, driver's facial features, and vehicle behavior features, to fuse information. These three MCSVM classifiers output probabilities as inputs for the three real-time dynamic basic probability assignments (BPAs) at the second level of fusion, labeled decision-level fusion. These BPAs, and the fusion result of the previous time step, are fused in the decision-level fusion based on the Dempster-Shafer evidence theory. This includes modifying the BPAs to accommodate the decision conflict among the different feature sources. Field experiments show that the proposed recognition method can outperform the single-fatigue-feature method and the single-source fusion-based method.