Real time conversion of sign language to speech and prediction of gestures using Artificial Neural Network

Sign language is generally used by the people who are unable to speak, for communication. Most people will not be able to understand the Universal Sign Language (unless they have learnt it) and due to this lack of knowledge about the language, it is very difficult for them to communicate with mute people. A device that helps to bridge a gap between mute persons and other people forms the crux of this paper. This device makes use of an Arduino Uno board, a few flex sensors and an Android application to enable effective communication amongst the users. Using the flex sensors, gestures made by the wearer is detected and then according to various pre-defined conditions for the numerous values generated by the flex sensors, corresponding messages are sent using a Global System for Mobile(GSM) module to the wearer's android device, which houses the application that has been designed to convert text messages into speech. The GSM module is also used to send the sensor inputs to a cloud server and these values are taken as input parameters into the neural network for a time series based prediction of gestures. The system is designed to be a continually learning device and improve reliability by monitoring every individual's behaviour at all times. (C) 2018 The Authors. Published by Elsevier B.V.