Multimodal System Based on Electrooculography and Voice Recognition to Control a Robot Arm

People with severe motor disorders cannot use their arms and/or hands to interact with the environment. In such cases, a human-machine interface can be used to allow these people to perform activities of daily life. Voice recognition would be the best method to control a device devoted to help with these tasks, but this method is very dependent on the background noise and, under certain circumstances, it cannot be used successfully. In this sense, other methods must be included in the system to assist the voice recognition module. In this project, electrooculography (EOG) has been selected due to its stability and robustness. This way, a multimodal system based on EOG and voice recognition has been developed to control a robotic arm. This paper presents the procedures designed to combine both methods to create a multimodal interface useful for disabled people. Tests presented in this document compare the skills of five different users while controlling the robotic arm to perform pick-and-place tasks. Task duration and accuracy have been measured to obtain specific scores that are used to evaluate both interaction methods independently and the multimodal combination of them. The system works successfully using both methods (EOG and voice recognition). In addition, the multimodal interface improves robustness and reduces the uncertainty generated by the environment when there is background noise.