Real-time sign language detection: Empowering the disabled community

被引:0
作者
Kumar, Sumit [1 ]
Rani, Ruchi [2 ]
Chaudhari, Ulka [2 ]
机构
[1] Symbiosis Int Deemed Univ, Symbiosis Inst Technol, Pune Campus, Pune 412115, Maharashtra, India
[2] Dr Vishwanath Karad MIT World Peace Univ Pune, Sch Comp Engn & Technol, Dept Comp Engn & Technol, Pune 411038, Maharashtra, India
关键词
Sign Language (SL); Disabled; Transfer learning; Convolutional neural networks (CNNs); VGG16; model; Pre-trained models; Classification;
D O I
10.1016/j.mex.2024.102901
中图分类号
O [数理科学和化学]; P [天文学、地球科学]; Q [生物科学]; N [自然科学总论];
学科分类号
07 ; 0710 ; 09 ;
摘要
Interaction and communication for normal human beings are easier than for a person with disabilities like speaking and hearing who may face communication problems with other people. Sign Language helps reduce this communication gap between a normal and disabled person. The prior solutions proposed using several deep learning techniques, such as Convolutional Neural Networks, Support Vector Machines, and K-Nearest Neighbors, have either demonstrated low accuracy or have not been implemented as real-time working systems. This system addresses both issues effectively. This work extends the difficulties faced while classifying the characters in Indian Sign Language(ISL). It can identify a total of 23 hand poses of the ISL. The system uses a pre-trained VGG16 Convolution Neural Network(CNN) with an attention mechanism. The model's training uses the Adam optimizer and cross-entropy loss function. The results demonstrate the effectiveness of transfer learning for ISL classification, achieving an accuracy of 97.5 % with VGG16 and 99.8 % with VGG16 plus attention mechanism. center dot Enabling quick and accurate sign language recognition with the help of trained model VGG16 with an attention mechanism. center dot The system does not require any external gloves or sensors, which helps to eliminate the need for physical sensors while simplifying the process with reduced costs. center dot Real-time processing makes the system more helpful for people with speaking and hearing disabilities, making it easier for them to communicate with other humans.
引用
收藏
页数:10
相关论文
共 50 条
[41]   A framework for metamorphic malware analysis and real-time detection [J].
Alam, Shahid ;
Horspool, R. Nigel ;
Traore, Issa ;
Sogukpinar, Ibrahim .
COMPUTERS & SECURITY, 2015, 48 :212-233
[42]   Real-time Object Detection and Tracking in Video Sequences [J].
Dornaika, F. ;
Chakik, F. .
INTELLIGENT ROBOTS AND COMPUTER VISION XXVII: ALGORITHMS AND TECHNIQUES, 2010, 7539
[43]   Real-Time Plant Disease Detection: A Comparative Study [J].
Singh, Yogendra ;
Shukla, Swati ;
Mohan, Nishant ;
Parameswaran, Sumesh Eratt ;
Trivedi, Gaurav .
AGRICULTURE-CENTRIC COMPUTATION, ICA 2023, 2023, 1866 :210-224
[44]   Real-Time Constrained Energy Minimization for Subpixel Detection [J].
Chang, Chein-I ;
Li, Hsiao-Chi ;
Song, Meiping ;
Liu, Chunhong ;
Zhang, Lifu .
IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING, 2015, 8 (06) :2545-2559
[45]   Real-time computer vision-based gestures recognition system for bangla sign language using multiple linguistic features analysis [J].
Rahaman, Muhammad Aminur ;
Ali, Md. Haider ;
Hasanuzzaman, Md. .
MULTIMEDIA TOOLS AND APPLICATIONS, 2023, 83 (8) :22261-22294
[46]   A real-time traffic sign detection in intelligent transportation system using YOLOv8-based deep learning approach [J].
Tang, Mingdeng .
SIGNAL IMAGE AND VIDEO PROCESSING, 2024, 18 (8-9) :6103-6113
[47]   The Coordinating Role of Language in Real-Time Multimodal Learning of Cooperative Tasks [J].
Petit, Maxime ;
Lallee, Stephane ;
Boucher, Jean-David ;
Pointeau, Gregoire ;
Cheminade, Pierrick ;
Ognibene, Dimitri ;
Chinellato, Eris ;
Pattacini, Ugo ;
Gori, Ilaria ;
Martinez-Hernandez, Uriel ;
Barron-Gonzalez, Hector ;
Inderbitzin, Martin ;
Luvizotto, Andre ;
Vouloutsi, Vicky ;
Demiris, Yannis ;
Metta, Giorgio ;
Dominey, Peter Ford .
IEEE TRANSACTIONS ON AUTONOMOUS MENTAL DEVELOPMENT, 2013, 5 (01) :3-17
[48]   Real-Time Motor Fault Detection by 1-D Convolutional Neural Networks [J].
Ince, Turker ;
Kiranyaz, Serkan ;
Eren, Levent ;
Askar, Murat ;
Gabbouj, Moncef .
IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS, 2016, 63 (11) :7067-7075
[49]   APELID: Enhancing real-time intrusion detection with augmented WGAN and parallel ensemble learning [J].
Vo, Hoang V. ;
Du, Hanh P. ;
Nguyen, Hoa N. .
COMPUTERS & SECURITY, 2024, 136
[50]   Real-Time Fatigue Detection Algorithms Using Machine Learning for Yawning and Eye State [J].
Makhmudov, Fazliddin ;
Turimov, Dilmurod ;
Xamidov, Munis ;
Nazarov, Fayzullo ;
Cho, Young-Im .
SENSORS, 2024, 24 (23)