Interact With Your Car: A User-Elicited Gesture Set To Inform Future In-Car User Interfaces

In recent years, stereoscopic 3D (S3D) displays have shown promising results on user experience, for navigation, and critical warnings when applied in cars. However, previous studies have only investigated these displays in non-interactive use-cases. So far, interacting with stereoscopic 3D content in cars has not been studied. Hence, we investigated how people interact with large S3D dashboards in automated vehicles (SAE level 4). In a user-elicitation study (N=23), we asked participants to propose interaction techniques for 24 referents while sitting in a driving simulator. Based on video recordings and motion tracking data of 1104 proposed interactions containing gestures and other input modalities, we grouped the gestures per task. Overall, we can report a chance-corrected agreement rate of kappa = 0.232 and by that, a medium agreement among participants. Based on the agreement rates, we defined two sets of gestures: a basic and a holistic version. Our results show that participants intuitively interact with S3D dashboards and that they prefer mid-air gestures that either directly manipulate the virtual object or operate on a proxy object. We further compare our results with similar results in different settings and provide insights on factors that have shaped our gesture set.