Wearable Sensors-Based Hand Gesture Recognition for Human-Robot Collaboration in Construction

The development of robotic machines has shown the potential to promote automation in construction. One of the critical enablers of human-robot collaboration is a user-friendly interface to support their interactions. Compared with other interfaces, hand gesture is an effective communication channel on construction sites. This article proposes a system for recognition of construction workers' hand gestures using wearable sensors on fingers. The system starts with synchronizing, normalizing, and smoothing finger motions. Then, the motion data are extracted through a sliding window and fed into an enhanced fully convolutional neural network (FCN) for the hand gesture recognition. The system was tested through a system validation test and achieved the precision and recall of 85.7% and 93.8%, respectively. A pilot study demonstrated the use of the proposed system to interact with a robotic dump truck. The system was further compared with vision-based recognition methods to quantitatively and qualitatively assess their relative benefits and limitations.