The use of visual and auditory feedback for assembly task performance in a virtual environment

This paper presents our creation and evaluation of multi-modal interface for a virtual assembly environment. It involves implementing an assembly simulation environment with multi-sensory feedback (visual and auditory), and evaluating the effects of multimodal feedback on assembly task performance. This virtual environment experimental platform brought together complex technologies such as constraint-based assembly simulation, optical motion tracking technology, and real-time 3D sound generation technology around a virtual reality workbench and a common software platform. A peg-in-a-hole and a Sener electronic box assembly tasks have been used as the task cases to perform human factor experiments, using sixteen subjects. Both objective performance data (task completion time, and human performance error rates) and subjective opinions (questionnaires) have been gathered from this experiment.