Emotional Intelligence for the Decision-Making Process of Trajectories in Collaborative Robotics

In collaborative robotics, to improve human-robot interaction (HRI), it is necessary to avoid accidental impacts. In this direction, several works reported how to modify the trajectories of collaborative robots (cobots), monitoring the operator's position in the cobot workspace by industrial safety devices, cameras, or wearable tracking devices. The detection of the emotional state of the operator could further prevent possible dangerous situations. This work aimed to increase the predictability of anomalous behavior on the part of human operators by the implementation of emotional intelligence (EI) that allows a cobot to detect the operator's Level of Attention (LoA), implicitly associated with the emotional state, and to decide the safest trajectory to complete a task. Consequently, the operator is induced to pay due attention, the safety rate of the HRI is improved, and the cobot downtime is reduced. The approach was based on a vision transformer (ViT) architecture trained and validated by the Level of Attention Dataset (LoAD), the ad hoc dataset created and developed on facial expressions and hand gestures. ViT was integrated into a digital twin of the Omron TM5-700 cobot, suitably developed within this project, and the effectiveness of the EI was tested on a pick-and-place task. Then, the proposed approach was experimentally validated with the physical cobot. The results of the simulation and experimentation showed that the goal of the work was achieved and the decision-making process can be successfully integrated into existing robot control strategies.