Facial Expressions as Indicator for Discomfort in Automated Driving

Driving comfort is considered a key factor for broad public acceptance of automated driving. Based on continuous driver/passenger monitoring, potential discomfort could be avoided by adapting automation features such as the driving style. The EU-project MEDIATOR (mediatorproject.eu) aims at developing a mediating system in automated vehicles by constantly evaluating the performance of driver and automation. As facial expressions could be an indicator of discomfort, a driving simulator study has been carried out to investigate this relationship. A total of 41 participants experienced three potentially uncomfortable automated approach situations to a truck driving ahead. The face video of four cameras was analyzed with the Visage facial feature detection and face analysis software, extracting 23 Action Units (AUs). Situation-specific effects showed that the eyes were kept open and eye blinks were reduced (AU43). Inner brows (AU1) as well as upper lids (AU5) raised, indicating surprise. Lips were pressed (AU24) and stretched (AU20) as sign for tension. Overall, facial expression analysis could contribute to detect discomfort in automated driving.