An Approach to Sri Lankan Sign Language Recognition Using Deep Learning with MediaPipe

Currently, there are millions of people around the world with speech and hearing impairments, and according to the latest census, there are nearly 70,000 people who use Sri Lankan Sign Language. Sign language is a visual language, and it is the main medium of communication in their daily conversations. But they face obstacles when communicating with people who do not know sign language. There are communication barriers in different contexts, such as in work environments, knowledge exchange, and message sharing. Therefore, technology should play a major role in helping people with these hearing and speech impairments to improve their quality of life. This research aims to suggest models that use Google MediaPipe Hand Pose landmarks to identify Sri Lankan Sign Language. Moreover, this article compares vision-based approaches with convolutional neural network (CNN) and recurrent neural network (RNN). We also considered activation functions (such as ReLU, Linear, and Softmax), loss functions (mean squared error (MSE) and Categorical_crossentropy), and optimizations (Adam and Stochastic Gradient Descent (SGD)). The result showed that most algorithms built with Long Short-Term Memory (LSTM), CNN, and CNN-LSTM achieved an accuracy greater than 95%, both with the training dataset and the test dataset. In particular, models with MSE as the loss function and Adam as the optimizer showed higher accuracy.