Virtual Reality Robot-Assisted Welding Based on Human Intention Recognition

We propose an innovative approach to enhance welding operations by using a cyber-physical system (CPS) with layered architecture and enabling a robot to be effectively operated by its commanding human. This article focuses on the recognition of the commanding human's intention that should be executed by the robot. To this end, a virtual reality (VR) system based on the HTC Vive is used to create a remote virtual welding environment. Human hand movement speed data is collected and used to train a hidden Markov model (HMM) using the Baum-Welch algorithm. The Bayesian information criterion (BIC) is applied to determine the number of hidden states. The state occupancy probability distribution (the probability of each state at a given time) is estimated based on the human hand movement speed sequence using the forward algorithm. The human intention, defined as the intended movement in this article, is then estimated as the statistical expectation of the observable variables. Using the proposed human intention estimation algorithm, the intended movement recognized from the raw movement data is smoother, which is preferable in welding tasks. A 6-DoF industrial robot, UR-5 with a custom gas tungsten arc welding (GTAW) torch installed, works as the final performer of the welding jobs. The robot receives the intended movement data from the HMM and uses this to assist the human welding operators. Welding experiments have been conducted both with and without the proposed human intention recognition (IR) algorithm. The results show that the robot can help the operators complete welding tasks with better performance using the proposed IR system, supporting the effectiveness of the proposed VR robot-assisted welding system. Note to Practitioners-Welding is not only labor-intensive but also requires real-time adaption to the process, which is challenging for robots/machines but relatively straightforward for humans. Using a human commanded robot to perform welding can liberate humans from laborious operations and hazardous environments. To this end, a virtual reality (VR) system is used to create a virtual welding environment for a human to view the process remotely and for a human to pass his/her resultant adaptation to the robot through hand movements. However, hand movements do not always fully represent the intended adaptation of the commanding human, and the recognition of such human intention is fundamental in such a proposed method. This article established the mathematical framework for the recognition of the human intention and, thus, the foundation for effective assistance of robots to humans.