Monitoring mouth movement for driver fatigue or distraction with one camera

This paper proposed to locate and track a driver's mouth movement using a dashboard-mounted CCD camera. Study on monitoring and recognizing a driver's yawning fatigue state and distraction state due to talking or conversation. Firstly determining the interest of area for mouth by detecting face using color analysis, then segmenting skin and lip pixels by Fisher classifier, and detecting driver's mouth and extracting lip features by connected component analysis, tracking driver's mouth via Kalman filtering in real time. Taking the mouth region's geometric features to make up an eigenvector as the input of a BP ANN, then we acquire the BP ANN output of three different mouth states that represent normal, yawning or talking state respectively. The experiment results show that this new method can inspect the driver's mouth region accurately and quickly, and will give a warning sign when it find driver's yawning fatigue state and distraction state due to talking or conversation.