Driver fatigue state detection method based on multi-feature fusion

被引：0
|
作者：
Fang H.-J. [1 ]
Dong H.-Z. [1 ]
Lin S.-X. [1 ]
Luo J.-Y. [2 ]
Fang Y. [2 ]
机构：
[1] ITS Joint Research Institute, Zhejiang University of Technology, Hangzhou
[2] Hangzhou Jintong Technology Group Limited Company, Hangzhou
来源：
Zhejiang Daxue Xuebao (Gongxue Ban)/Journal of Zhejiang University (Engineering Science) | 2023年 / 57卷 / 07期
关键词：
driver safety; driving simulation; fatigue detection; video analytics; YOLOv5;
D O I：
10.3785/j.issn.1008-973X.2023.07.003
中图分类号：
学科分类号：
摘要：
The improved YOLOv5 object detection algorithm was used to detect the facial region of the driver and a multi-feature fusion fatigue state detection method was established aiming at the problem that existing fatigue state detection method cannot be applied to drivers under the epidemic prevention and control. The image tag data including the situation of wearing a mask and the situation without wearing a mask were established according to the characteristics of bus driving. The detection accuracy of eyes, mouth and face regions was improved by increasing the feature sampling times of YOLOv5 model. The BiFPN network structure was used to retain multi-scale feature information, which makes the prediction network more sensitive to targets of different sizes and improves the detection ability of the overall model. A parameter compensation mechanism was proposed combined with face keypoint algorithm in order to improve the accuracy of blink and yawn frame number. A variety of fatigue parameters were fused and normalized to conduct fatigue classification. The results of the public dataset NTHU and the self-made dataset show that the proposed method can recognize the blink and yawn of drivers both with and without masks, and can accurately judge the fatigue state of drivers. © 2023 Zhejiang University. All rights reserved.
引用
收藏
页码：1287 / 1296
页数：9
相关论文
共 26 条
  • [1] HOODA R, JOSHI V, SHAH M., A comprehensive review of approaches to detect fatigue using machine learning techniques [J], Chronic Diseases and Translational Medicine, 8, 1, pp. 26-35, (2022)
  • [2] KUWAHARA A, NISHIKAWA K, HIRAKAWA R, Et al., Eye fatigue estimation using blink detection based on eye aspect ratio mapping(EARM) [J], Cognitive Robotics, 2, 1, pp. 50-59, (2022)
  • [3] ZHANG K, ZHANG Z, LI Z, Et al., Joint face detection and alignment using multitask cascaded convolutional networks [J], IEEE Signal Processing Letters, 23, 10, pp. 1499-1503, (2016)
  • [4] GUO X, LI S, YU J, Et al., PFLD: a practical facial landmark detector
  • [5] ZHANG Z, PING L, CHEN C L, Et al., Facial landmark detection by deep multi-task learning [C], European Conference on Computer Vision, pp. 94-108, (2014)
  • [6] CHEN L, XIN G, LIU Y, Et al., Driver fatigue detection based on facial key points and LSTM [J], Security and Communication Networks, 2021, 8, pp. 1-9, (2021)
  • [7] WANG J, YU X, LIU Q, Et al., Research on key technologies of intelligent transportation based on image recognition and anti-fatigue driving [J], EURASIP Journal on Image and Video Processing, 2019, 1, pp. 1-13, (2019)
  • [8] ADHINATA F D, RAKHMADANI P, WIJAYANTO D., Fatigue detection on face image using FaceNet algorithm and K-nearest neighbor classifier [J], Journal of Information Systems Engineering and Business Intelligence, 7, 1, pp. 22-30, (2021)
  • [9] PHAM L, VU T H, TRAN T A., Facial expression recognition using residual masking network [C], 25th International Conference on Pattern Recognition, pp. 4513-4519, (2021)
  • [10] LIU M Z, XU X, HU J, Et al., Real time detection of driver fatigue based on CNN-LSTM [J], IET Image Processing, 16, 2, pp. 576-595, (2022)
123