Real-time Arabic Sign Language Recognition based on YOLOv5

Sign language is the most common communication mode of deaf and mute community. However, hearing people do not generally know this language. So, an automatic sign langage recognition is required to facilitate and better understand interactions with such people. However, one of the main challlenges in this field is the real-time sign recognition. That is why. deep learning-based object detection models can be used to improve the recognition performance (in terms of time and accuracy). In this paper, we present a real-time system that allows the detection and recognition of hand postures intended for the Arabic sign language alphabet. To do so, we constructed a dataset of 28 Arabic signs containing around 15,000 images acquired with different sizes of hands, lighting conditions, backgrounds and with/without accessories. We then trained and tested different variants of YOLOv5 on the constructed dataset. The conducted experiments on our ArSL real-time recognition system show that the adapted YOLOv5 is more effective than Faster R-CNN detector.